Kedokumango

Visão geral

  • Data de fundação 15 de maio de 1925
  • Setores Indústria alimentícia

Descrição da Empresa

Open-R1: a Totally Open Reproduction Of DeepSeek-R1

Hey there! This post is an introduction to the job, not a claim that we’ve reproduced R1 yet. We’re integrating in the open, so as quickly as we have evaluation numbers, we’ll share them. You can follow our development on Hugging Face and GitHub.

True, but it seems like there’s absolutely nothing to be assessed since today. I presume the ultimate goal is to train a brand-new thinking model and then use the very same assessment metrics as o1 and the DeepSeek-R1.

Well, there ought to be at least some peace of mind check and recognition to ensure the model was trained properly.

Oh yes, if you are discussing the evaluation number of deepseek’s model it’s coming soon!

As mentioned in the blog site post there is no model called Open-R1 to check at all … not yet anyhow. This is a that Hugging face will take the R1 Deepseek model, work out how it was built as described in the paper and from what they launched, and then reproduce that procedure.

in reality this is basically how science works … A creates a plan, discovery or development and it is evaluated by B, C and D to see if it is reproduceable. Thats been the foundation of research now for a few centuries.

This blog is not saying they have actually already done so … Its a blog site outlining an intent to begin training a design like R1 and calling it Open-R1.

Also DeepSeek-R1 was just released recently, and even in their paper they detailed the calculate hours needed. While those are low calculate hours for a SOTA design this does not indicate you can train stated model in a week. I ‘d personally enjoy to be able to train a transformer design in a week, but we may need to wait a while for that level of calculate innovation.

So there are no standards for a design that has not been developed yet right? As laid out in the blog site, and once again in reply to your question.

However fear not, there is a GitHub Repo currently and factors (hell I might join myself), some prelim work done, and a master plan. A great starting position.

n
@edbeeching
has assessed the released designs already

( src: https://x.com/edwardbeeching/status/1884273209136275742)

R1 simply trained on o1 outputs, so collectively …/ s. This is what the new AI czars are saying

Hi! This article is an introduction to the project, not a claim that we have actually recreated R1 yet. We will completely share the missing out on piece when we have them, you can expect the models and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

That’s nice and essential to understand this remarkable hype that does not have technical understanding and explanation. Science has to do with reproduction, and if they declare to be open, let them fullfill the open part.

Please do publish the training expense.

We will!

Excalidraw Hi n
@bojan2501
thanks, we will undoubtedly be working hard to make certain this training recipe can work for little language models on consumer hardware since not everybody has a cluster of H100s in the house:-RRB- The tool we used for the images was Excalidraw! https://excalidraw.com

eagerly anticipating it! WTF are your speaking about?

must be a joke

It’s truly cool to see how the entire open source community comes together!

Ops …

5.5 M is number press reporter in the deepseekv3 tech report (just the training, not the experiment afaik), for R1 hard to approximate tbh but much less than 5.5 M imo

Historically, they have never launched code or datasets of their LLM training, so I wouldn’t expect this time to be different. If they would release it that would be amazing of course!

Yes naturally!

So basically you’re asking to replace existing censorship with another flavour of censorship?

The code for the models are inside the design repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py

Hello Team, I’m Ray Bernard, the author and creator of EQUATOR. My research study group will be working on a paper concentrated on reproducing certain elements of DeepSeek R1. Our goal is to recreate the cold start and supply your group with a dataset that consists of COT and other techniques to support these efforts. We like to contribute our work to assist. Please let me understand if you find this useful. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/

Where is the examination numbers? without it you can’t call it reproduction.

8 replies

True, but it looks like there’s absolutely nothing to be examined as of right now. I assume the supreme objective is to train a new reasoning model and then utilize the very same assessment metrics as o1 and the DeepSeek-R1.

That’s rather fascinating, I was asking myself why the concerns the author exposed here are not being asked by others? I think the work they have done is remarkable but at the same time I wonder why they wouldn’t put these missing pieces on if they are expected to be completely open.
Why even without recreation and understanding of the innovation they could affect a lot the market in this method?

4 replies

Hi! This post is an intro to the task, not a claim that we have actually recreated R1 yet. We will totally share the missing out on piece when we have them, you can anticipate the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo

Interesting read, and it is excellent that we see more effort into this direction: more optimization and less strength.
Also wonder what tool did the author usage for creating action diagram.

2 replies

Excalidraw I’m so grateful that initiative like this already exist, I’m gon na try to contribute:-RRB- 1 reply

looking forward to it! So racist articel

2 replies

WTF are your speaking about?

Awesome to have this open recreation began!

For Step # 1 check out https://github.com/open-thoughts/open-thoughts!

https://x.com/ryanmart3n/status/1884284101265612856

Let’s do this thing!

1 reply

It’s truly cool to see how the entire open source neighborhood comes together!

Does anyone know the real training expense of r1? I can’t find it in the paper or the announcement post. Is the 6M cost reported by media simply the number taken from v3’s training expense?

2 replies

Ops …

Has anybody asked the DeepSeek group to publish their training data and code, or at least share them privately with an independent duplication job like this? Have they turned down such a demand?

A loyal replication depends on using the very same dataset and hyperparameters. Otherwise, any major disparities with the published criteria would be hard to pin down-whether due to training information differences or the duplication method itself.

1 reply

Historically, they have never launched code or datasets of their LLM training, so I wouldn’t expect this time to be various. If they would launch it that would be incredible naturally!

In the meantime we need to make finest guess price quotes and see if we can arrive ourselves.

You supply great duplication procedure of Deepseek thinking training. I will try something similar to it.

This is truly good details, can we tweak with specific usage case when code is launched?

1 reply

Yes naturally!

Please consider getting rid of biased, tainted or unaligned training information and make an effort to remove copyrighted works from the crawl from consumption. This will make the design more functional. If you recycled anthropic curation checks, this may likewise assist, remove obviouslybiased data will likely include a great deal of worth. We do not desire another tainted, unaligned open source design, right? And no business would ever use deepseek or a model that reuses it, right?
We value your work for the benefit of humanity, we hope.
Miike C from NJ

1 reply

So generally you’re asking to replace existing censorship with another flavour of censorship?

Can’t wait! Hopefully the model will be uncensored however whatever you can do is alright! Love seeing open source building itself up. I’m not smart sufficient to in fact assist but I can contribute ethical assistance lol

Hello guys, I am even simply searching for code for DeepSeek-V2, in order to fully comprehend multi-head latent attention. You do not appear to have code in Hugging Face even for that. Or am I missing out on something? Don’t see anything in src/transformers/models. MLA is not correctly explained in their paper, so it would be essential to have code for this.

💬 Dúvidas?
Escanear o código