Reflection AI Unveils Open-Weight Beam MoE: 501B Params with 23B Active, Cutting Inference Compute to a Quarter of GLM-5.2

Reflection AI has released an open-source MoE model named Beam (501B total parameters, 23B active), focusing on coding and reasoning tasks. It approaches GLM-5.2 performance on advanced reasoning benchmarks while requiring only one-quarter of the inference compute; model weights are scheduled for release in October. NVIDIA is currently in early-stage negotiations regarding investment or acquisition.

Beam differentiates itself through superior inference efficiency; its MoE architecture supports high-throughput, low-latency agentic scenarios; and its open-weight strategy directly targets China’s leading open-source model ecosystem.

Target Audience: Large Model Architects, AI Infrastructure Engineers, and Open Source Model Strategy Leads.

Translation | Lin Qibei Editing | Cai Fangfang

Recently, Reflection AI released its first open-weight model, Beam, claiming to benchmark against Chinese open-source models. Less than a week later, the company—dubbed by netizens as the "American DeepSeek"—was reported to be under consideration for acquisition by NVIDIA.

According to the Financial Times, both parties are negotiating further investment or acquisition possibilities. Potential deal structures also include hiring the team and licensing the technology. Currently, negotiations remain in the early stages with no agreement reached, and neither party has publicly responded to the report.

As a key investor in Reflection AI, NVIDIA’s pursuit of deeper collaboration has drawn increased attention to this startup focused on open models and its technical roadmap.

Reflection AI is a startup dedicated to building frontier open models, founded in 2024 by former Google DeepMind researchers Misha Laskin and Ioannis Antonoglou. The company’s first open-weight model, Beam, announced in early October, directly challenges the leading position of Chinese models in the open ecosystem.

Beam employs a sparse Mixture-of-Experts (MoE) architecture with 501 billion total parameters and 23 billion active parameters, targeting coding, reasoning, and agentic tasks. According to evaluations published by Reflection AI, Beam achieves performance comparable to GLM-5.2 on certain advanced reasoning benchmarks, requiring approximately one-third to one-quarter of the inference compute. Against more capable open models like Kimi K3, its primary competitive advantage lies in inference efficiency. As planned at launch, model weights and accompanying technical documentation will be released within October, with early access applications currently available.

On October 10, Reflection AI Co-founder and CEO Misha Laskin appeared on the tech podcast NO PRIORS. He argued that open models will become critical AI infrastructure, and that reinforcement learning, enterprise customization, and open ecosystems will jointly drive the next phase of development.

Regarding China’s leadership in open models, Misha described it as a "gift" to global developers. One reason Reflection AI decided to conduct pre-training in-house was that the team needed a robust open foundation but found that the best options primarily came from China. He believes Chinese models will remain competitive, but advocates for a diverse, multi-polar competitive ecosystem for open models. Furthermore, competition extends beyond model capability: when enterprises and nations build applications around a specific model, they may gradually adopt the associated chips, software, and infrastructure.

Competitive open models could influence future compute deployment and hardware/software choices. Following this logic, NVIDIA’s interest in Reflection AI adds another dimension worth considering.

TL;DR:

Q: Why did Reflection AI shift from reinforcement learning to training full open models themselves?

A: Initially, we wanted to use reinforcement learning on existing open models to enable agentic capabilities. However, we discovered that the best open models were primarily from China, and we needed a strong model foundation. Meanwhile, research indicated that achieving good results with large-scale reinforcement learning requires doing your own pre-training; the two are closely linked.

Q: What is the hardest part about building a frontier model lab from scratch?

A: Everything is hard. You have to get everything right: recruiting and retaining talent, securing data and compute, and building infrastructure so the team works toward the same goal. At DeepMind, many tools were mature enough that researchers took them for granted; after starting up, we had to build these from scratch. We grew from about 30 people to 300 in roughly a year before gradually acquiring full model R&D capabilities.

Q: Is catching up with frontier models necessarily dependent on having as many resources as large labs?

A: Not necessarily during the catch-up phase. Research can be divided into exploring new ideas and implementing known effective methods. Top-tier talent can reduce unnecessary exploration and improve capital efficiency. However, reaching the true frontier requires commensurate resource scale. Capital investment eventually hits limits, but fortunately, compute efficiency is improving, and models themselves are helping us build better models.

Q: Which application directions do you see creating economic value?

A: Coding and agentic capabilities are foundational, but model abilities are not uniform; switching tasks or execution frameworks doesn’t always result in smooth adaptation. With appropriate data, however, performance improves quickly. Therefore, the key is finding valuable business cases, establishing good evaluations, and using synthetic data and customization to achieve a better balance between cost and effectiveness—not necessarily training directly on customer data.

Q: How do you commercialize open models?

A: Both open and closed-source models aim to expand inference businesses. Permissive model licenses don’t mean deployment is free: customers still need inference software, cluster management, execution frameworks, and compute. We help enterprises set up these supporting components, turn them into valuable applications, and let those applications drive sustained inference demand. The value of our service lies in helping customers actually use the technology.

Q: How will enterprises transition from closed-source to open models?

A: Typically, they start by "renting intelligence" before considering "owning intelligence." Enterprises usually run significant workloads on closed-source models before considering self-deployment due to cost and control concerns. But I believe most enterprises will first customize systems—for example, building agentic workflows suited to their business around the model—and only gradually fine-tune the model once applications and data accumulation mature.

Q: What is your view on China’s leadership in open models?

A: This is beneficial for the world; it can be seen as a gift. Many Western companies can thus build more sustainable businesses and reduce dependence on closed-source vendors. I believe Chinese models will continue to excel, but the open model ecosystem should maintain diverse competition. The West also needs to establish a competitive ecosystem to offer the market more choices and competition.

Q: Why does competition among open models involve geopolitics?

A: A model alone is not enough; users also need supporting software and infrastructure. Once a country adopts a particular model, it may gradually adopt the entire accompanying technology stack, becoming deeply tied to the supplier’s supply chain. Consequently, the impact of models extends to chips and data centers. Every nation wants other countries to build products on top of its own technology, which confers both commercial and geopolitical advantages.

Q: Are open models less secure?

A: This concern is valid, but security cannot be understood solely through a closed-source lens. It is difficult for a few hundred researchers in a handful of labs to cover all potential vulnerabilities. I believe that involving more researchers in discovering and patching issues helps improve security. Especially in cybersecurity, offensive and defensive capabilities are hard to separate; removing offensive capabilities may also deprive those willing to participate in defense of their tools.

Q: What do you think AI safety should focus on most?

A: We need to distinguish between risks that have already materialized or are foreseeable in the near term, and more theoretical scenarios. I tend to prioritize real-world problems. There is no magic equation for alignment; much of the work involves identifying flaws and then fixing them one by one with data and algorithms. As systems become more powerful, we must certainly pay attention to unintended consequences, but this also means requiring more people to participate in actual detection and remediation.

Q: If AI can accelerate scientific research, will we still need so many researchers and engineers in the future?

A: Completing the same tasks might require fewer people, but the scope of what we want to achieve is expanding, so we still feel understaffed. Models can speed up computation, experimentation, and infrastructure work, but humans still need to judge which parts can be delegated to AI and which require creativity. Greater demand will arise at the application layer: enterprises need engineers who understand models, know how to evaluate them, and can go on-site to solve practical problems. This type of talent still needs significant cultivation.

How Much Compute and How Many Things Done Right Does It Take to Catch Up with Frontier Models?

Host: Joining us today is Misha Laskin, Co-founder and CEO of Reflection AI. Reflection is dedicated to providing open-weight models, aiming to drive the future development of intelligence. Previously, Misha served as a researcher at Google DeepMind and holds a PhD in physics. Misha, could you first introduce your latest progress? You’ve been doing very interesting work on open-weight and open-source models, particularly focusing on advancing these efforts in the United States. Could you talk about what you’ve been working on recently and what you’re building?

Misha Laskin: Over the past 12 months, we established our company mission: to build frontier open intelligence and make it accessible to as many people as possible. In reality, we’ve been sprinting full tilt during this period to establish a lab truly capable of achieving this goal.

This reminds me of something Reid Hoffman might have said: Building the plane while flying it. Our experience has indeed been like that. About a year ago, we had perhaps only around 30 people, but creating such models genuinely requires over a hundred, even several hundred, researchers and engineers. It’s truly a project comparable to building a rocket. Now we have approximately 300 people. Teams across pre-training, mid-training, and reinforcement learning have been formed, and we have trained our first batch of models end-to-end. We just released a model named Beam, which is Reflection’s first open model.

Host: What was the most challenging part of this process? When discussing scaling models or building them from scratch, people usually mention several issues. First is compute—how to acquire sufficient compute, especially setting up a training cluster that can truly operate cohesively and effectively. Second is talent—clearly, companies are engaged in fierce compensation competition to recruit researchers. Then there is data scale. Among these factors, which posed the greatest constraint or was hardest to solve for you?

Misha Laskin: Interestingly, someone asked me the same question during an all-hands meeting at the company: “What was the hardest thing over the past year?” The reality is that every single aspect was extremely difficult. Someone might ask, “What exactly makes your models perform better than others?” The answer is again, every link in the chain. You essentially have to get 30 things right. That is precisely where the difficulty lies: you must excel at all 30 of those things. You need to hire excellent talent and retain them; you need a strong sense of mission and team culture so that everyone can collaborate, weather storms together, and strive toward the same direction.

You need to acquire data, acquire compute, and set up infrastructure to ensure that compute is actually usable. In mature large labs, many components have been built over years to form a relatively stable foundation. Having previously worked at DeepMind, the tools I took for granted—or rather, the tools I assumed would always work because they consistently did—are things I had to build from scratch here. So my answer is that everything is difficult; but being able to handcraft these solutions brings immense satisfaction.

Host: You mentioned earlier that about a year ago, you truly committed to doing this. Could you discuss how your goals and level of commitment changed, and what triggered this shift?

Misha Laskin: Before discussing what triggered this shift, I’d like to provide some background on founding the company two years ago. My co-founder Ioannis and I were involved in developing the first generation of Gemini models. We were both on the reinforcement learning team, led by Ioannis. At the time, we successively released Gemini 1 and Gemini 1.5. The user experience of models in that early stage was very similar to the original ChatGPT—it was primarily just chatting. They basically lacked true coding capabilities or agentic abilities; these models were merely good at conversation. They were predominantly pre-trained models, with 95% of compute spent on pre-training. We, however, focused on reinforcement learning, which was our path into research. My co-founder Ioannis was among the earliest founding engineers at DeepMind, participating in all major RL projects at the lab, including AlphaGo, and making key contributions.

Our judgment at the time was that since we were on the RL team, we believed that if we could apply RL effectively, we could unlock new capabilities.

In the first year after the company was founded, one of the first things that happened was that reinforcement learning (RL) started delivering results faster than we expected—both in our own experiments and across the industry. o1 was released during that period. We had originally assumed it would take longer, because from ImageNet in 2012 to RL systems actually working effectively at scale took about four years, so we thought this might require a similar timeline. But every year, actual progress has been faster than my predictions from the previous year.

About a year after founding the company, we realized that the best open models were coming from China; there weren't truly excellent open models from the West. And the technology we were developing required such a model as a foundation. Considering various factors and the geopolitical landscape at the time, we decided that the most impactful move was to build open models end-to-end ourselves, combining our bet on reinforcement learning with full-scale model development.

There were reasons related to enterprise demand, geopolitics, and research: we later discovered that if you want RL to work extremely well at scale, you really need to do your own pre-training. Both tasks must be done because they are too tightly coupled.

Host: The conventional wisdom has always been that pre-training at frontier model scale is prohibitively expensive, or at least extremely costly. Clearly, a lot of compute has now shifted toward post-training. How did you think about the resources needed and the allocation between different stages? Did you have certain model design principles that made you feel this was feasible from the start? Or did you just decide, "This must be done, and we'll figure out the resources later"?

Misha Laskin: One of the benefits of being an open model lab is that these things can be discussed publicly. To produce something meaningful does require significant resources, but it doesn't mean you need frontier-lab-scale resources right away. You only need resources comparable to other frontier labs once you're truly at the cutting edge of intelligence development.

You can divide research into two parts: exploring new ideas and executing known effective methods. The reason top talent is so important is partly because they reduce unnecessary exploration, allowing you to focus on executing what works. If the goal is to catch up to the frontier, you can do so with much higher capital efficiency.

Host: How much funding do you think is needed to catch up to the frontier?

Misha Laskin: I think it depends, because the frontier keeps moving forward. In fact, the amount of funding required increases every year. I'll return to the resource allocation between pre-training and post-training later. About a year ago, or 18 months ago, the required funding was probably in the hundreds of millions of dollars range. Now, even six months ago, it was likely already in the billions, though not yet tens of billions. By next year, I believe it will enter the tens-of-billions range.

You can understand it this way: for each generation of model iteration, the compute scale roughly quadruples. I have a rough heuristic: look at the state-of-the-art chips at the time. A few years ago, it might have been H100s, perhaps at a scale of 100,000 H100s. Later, training Astra used 100,000 Blackwells for a large-scale run, which represented roughly another 4x increase in compute compared to H100s. I believe the next generation will likely use 100,000 Vera Rubins. This gives you a general sense of how the scale changes: from hundreds of millions, to billions, to tens of billions.

Host: However, this growth seems like it must eventually approach a ceiling at some stage. Because relative to these companies' revenue scales, or even potential revenue scales, their ability to continue investing based on the economic value ultimately created will hit a limit. So, do you think model size, or at least training cluster size, is approaching this upper bound?

Misha Laskin: Absolutely. Capital expenditure has limits. Frontier labs have already invested hundreds of billions of dollars. Will they invest trillions in the next year or two? Probably not. However, several things are happening, one of which is that compute usage efficiency is improving dramatically because models themselves are becoming tools to help build models.

When research was done entirely by human researchers, efficiency improved by roughly 7x per year. These changes can be tracked using fairly rigorous technical methods. Take pre-training, for example: you have a pre-training loss, and you use it to measure improvements in compute efficiency. After making changes, you get a new pre-training loss curve; ideally, you should be able to reach the same level with less compute. So, the "7x" figure is actually a very specific, researchable metric. Looking at the system as a whole, pre-training may be relatively mature, but reinforcement learning still has huge room for improvement. Our current efficiency gains might already be reaching 30x, depending specifically on how strong the model is at improving itself. This is something happening right now.

Host: 30x per year, or over what timeframe?

Misha Laskin: Per model generation. It depends on exactly what metric you're measuring. But I do believe this speed is likely already 4x faster—or more—than when researchers worked alone, and it will probably continue to accelerate.

The amount of intelligence generated per floating-point operation (FLOP) is increasing, so even if compute scale approaches a ceiling at some stage, that's acceptable. Additionally, the revenue generated per FLOP is also increasing. The more intelligence contained within, the more value it can create. Returning to the question of how to allocate resources between pre-training and reinforcement learning: Beam has 500 billion total parameters and 23 billion active parameters, and its training utilized 6,000 GB300s. I remember it ran for several weeks back then, but now, with improved infrastructure efficiency, it takes about 12 days, or even less. Reinforcement learning used slightly more than 10,000 GB300s for four weeks. So, in practice, reinforcement learning consumes more FLOPs. RL tasks are also more complex because they involve massive inference at scale, configuring various sandboxes for agents, while simultaneously conducting training.

I entered the AI field because I saw my co-founder's work on AlphaGo. At the time, I was still a physicist, and there was a moment that impressed me deeply: a chart from AlphaGo showed its capabilities continuously improving without stopping. They finally stopped training at a certain point because what was the point of further improvement? It had already beaten the world champion. I thought then, if we could figure out how to apply this method to economically valuable tasks, the question becomes: how much money are you willing to invest to keep making this system better? That turns it into an economic problem.

The field has now reached this stage. What excites me most about this open-source project—and the technical report we will release alongside it—is that we will explain exactly how these systems were built. Our reinforcement learning (RL) system never stopped learning. If you look at the curves, they consistently trend upward; scaling further is essentially just a matter of compute. This means our field is now capable of building highly general-purpose RL systems whose capabilities continuously improve. Of course, you want to make them much more efficient, extracting as much intelligence as possible from existing compute resources. But this also implies that the problem has shifted from "can we train such a system?" to "where do we get the data, and what tasks have economic value?" Deciding how much money to invest for a certain degree of improvement ultimately becomes an economic decision.

Host: What directions are you planning to focus on next to further enhance model capabilities? Are there any developments you can share?

Misha Laskin: Some scientists might be reluctant to accept this framing, but AI development today, while still reliant on scientific research, increasingly involves solving engineering problems. It remains a science, but it is becoming more like an engineering discipline. Take rocketry as an analogy: there is a scientific aspect to building rockets, but most people view it as an engineering endeavor that includes some scientific research. That is roughly the current sentiment.

The main avenues for scaling today are relatively well-defined: primarily pre-training, synthetic data, and reinforcement learning. There may be entirely novel approaches we haven't discovered yet, and new labs are exploring those, but the current mainstream consists of these areas. Or, in a sense, it can be summarized as two directions: training and reinforcement learning.

Within these stages, there is still room for improvement in architecture, data, and algorithms. However, it is no longer the phase we saw five years ago, where bold exploration across various directions was the norm. One of the truly valuable aspects of DeepMind and Google Brain—including early OpenAI—was their diverse betting strategy. Everyone was trying bold and interesting things. Now, research directions have converged significantly. There is also an element of the "hardware lottery" at play: once a particular direction starts working, hardware co-optimizes around it. Consequently, even if you have a brilliant new idea, if it doesn't align well with existing hardware, it may not be cost-effective to implement, making it harder for new directions to emerge.

Returning to your question, there is no sign yet that improvements in pre-training are approaching a ceiling. We haven't seen compute efficiency gains slowing down; new developments continue to emerge. The potential for improvement in reinforcement learning is particularly large, and I believe there is still a lot to discover. However, even so, these discoveries feel more like engineering breakthroughs than foundational understandings of a completely new science.

From "Renting Intelligence" to "Owning Intelligence": How Do Open Models Become a Business?

Host: You mentioned that investments need to correspond to specific economic activities or yield productivity gains to financially sustain your work. As far as I know, Anthropic bet early on coding, and OpenAI clearly did the same. They initially had a coding model, partnered with GitHub Copilot, and later expanded into consumer markets and other domains. From the perspective of creating economic value, are there specific application areas you are focusing on? Is it coding and agentic workflows, or other fields? These seem to be the foundation for many applications. I am curious whether you have formed judgments about where models will be used most heavily, or which directions you need to steer them toward.

Misha Laskin: Yes. I believe coding and agentic capabilities lay a certain foundation for intelligence, and the final state of this intelligence is quite interesting: its capability distribution is uneven. On one hand, generalization occurs in unexpected ways. I was surprised by how quickly our models reached certain fairly new capability levels, which other labs only recently discovered. There is indeed some form of capability bridging and generalization happening here. On the other hand, performance exhibits noticeable variability. Once you have data suitable for the target task, the model adapts quickly; however, even between different benchmarks—for example, between different versions of Terminal-Bench—this inconsistency persists. Smooth generalization does not naturally occur across different execution harnesses. Nevertheless, as long as the system is already functional and possesses general agentic capabilities, providing it with data adapted to a new harness allows it to start adapting very quickly.

This means you already have a highly adaptable system; the next challenge is identifying where economically valuable data resources lie. I think this is also why open models are powerful. Enterprises and customers can take the model, customize it for their own business needs, and find near Pareto-optimal solutions for specific workloads—that is, achieving the highest possible performance at the lowest possible cost. This is a question that requires practical answers: you go to the customer, see what is actually valuable to them, and then determine if you can generate data for it. Typically, as long as something has economic value, corresponding data can be generated, because you don't necessarily need to train directly on customer data. More often, you are establishing evaluation methods; if you can design good evaluations for the customer's tasks, you can generate synthetic data that closely resembles them and achieve excellent generalization.

Specifically, various "Know Your Customer" (KYC) processes and compliance workflows in the financial sector are highly valuable. Many tasks in cybersecurity, especially network defense, are also quite valuable, as is the legal domain. Enterprises have various vertical businesses worth automating with agents, and moreover, the methodological patterns for getting things done in these fields are often very similar.

Host: Beam stands out particularly for its inference efficiency. Could you elaborate on that?

Misha Laskin: Beam is a model trained specifically for coding and agentic tasks, which are its strengths. When building a model, what matters is not just capability, but also the speed at which the agent completes a task. For customers, this means shorter task duration and lower costs.

If you ask a question via an AI chat tool and have to wait ten minutes for an answer, you naturally wish it could respond in one minute. Beam's efficiency is typically three to four times that of models with comparable capabilities; compared to larger models currently on the market, the efficiency advantage is even more pronounced, potentially reaching around ten times. It is so efficient because we first prioritize laying a solid foundation for reasoning capabilities through pre-training, then further enhance this capability using large-scale reinforcement learning. We believe this may be the largest-scale RL training in the open-source domain to date. At least, I haven't seen public records indicating anyone else has run RL continuously for four weeks on 10,000 GB300 GPUs.

The goal of reinforcement learning is to enable a system to achieve maximum capability in minimum time. Previous systems like AlphaGo underwent the same process. Early versions of AlphaGo took more convoluted paths when solving problems, but by the time it reached the level where it played against Lee Sedol, its search had become highly intelligent. That intelligence was what truly underpinned its capabilities. The more reinforcement learning is applied, the stronger the system becomes and the faster it solves problems.

Host: Great, that sounds very pragmatic.

Misha Laskin: I think this intuition stems from our belief in reinforcement learning. We have always believed that the path to Artificial General Intelligence (AGI) involves applying reinforcement learning on top of very strong foundations. People often forget that the earliest AlphaGo systems were also trained on game records from human amateur experts, so they first performed imitation learning before moving to reinforcement learning. Therefore, these two approaches need to complement each other. One benefit this brings is that you get a model with real economic value and high inference efficiency, which serves as a practical workhorse for enterprises and sovereign AI projects in the public sector.

Host: From a monetization perspective, open-source models and open-weight models have different commercialization paths. Companies like Mistral have been exploring various routes early on. Are you already advancing toward a specific direction? Is there anything you can share?

Misha Laskin: I believe that the commercialization of open models and closed-source models is, in a sense, quite similar. What you really want to do is scale your inference business as much as possible. The difference essentially lies in whether you are renting inference capacity or owning it.

For example, when you purchase a token, you are actually renting a small fraction of the entire technology stack. This includes the runtime framework—if there is an agentic runtime framework involved—as well as the model, inference software, cluster management software, and the GPUs running them. All these costs are amortized into a single token; what you rent is a slice of this entire ecosystem. When, for various reasons, you want to own your own intelligence, I compare it to housing: we usually start by renting apartments, but as life stages change, for various reasons, we may want to buy our own house, even though owning a home can sometimes be troublesome. But when your financial situation is mature enough, perhaps after starting a family, you will want to own it. AI in the commercial market has reached a stage where enterprises spend significant amounts renting intelligence, but then, due to considerations like control, begin to desire owning their own intelligence.

Returning to the question of monetization, what exactly is the difference between closed-source and open models? Open models have more permissive usage licenses, but for customers to truly utilize them, they still need all the surrounding infrastructure. When using closed-source models, the supporting elements such as cluster management software, inference software, and runtime frameworks are typically provided by the service provider; deploying open models requires the same support. We provide the complete suite of tools needed by large enterprises and sovereign AI clients to help them deploy open models. However, tools alone are not enough; most enterprises also need assistance with implementation. Therefore, we go deep into the customer's business context to help them realize truly valuable application scenarios. These scenarios generate more compute and inference demand. This is the business logic behind our services: helping customers effectively use open models, thereby driving growth in the inference business.

Host: Looking ahead one to two years, what do you think the approximate ratio of tokens processed by open models versus closed-source models will be?

Misha Laskin: I think the trend is already visible. About six months ago, on model gateway platforms like OpenRouter and Vercel, closed-source models accounted for the majority, while open models were the minority. Now, the ratio has almost completely flipped: from a 70/30 split favoring closed-source, it has shifted to a 70/30 split favoring open models. I believe this trend will only accelerate. I suspect the future landscape might closely resemble that of operating systems: over 95% of servers and computers worldwide run on open-source operating systems like Linux. This does not mean closed-source products lack high value—Microsoft and Apple are extremely valuable companies. Both open and closed ecosystems have room to grow, and the overall market is large enough.

I believe that the majority of token demand will flow to open models. There is a significant distinction here: I think substantial economic value will also flow to open models because, even if the model license is very permissive or carries certain constraints, the other components required to run it remain expensive. Compute is still costly, unlike in the CPU world. Hardware accelerators like GPUs represent a more expensive computational foundation than CPUs, necessitating the establishment of a different kind of cloud around them. I expect most tokens to flow to open-source models, while extremely valuable closed-source model companies will continue to exist in the market, alongside a valuable ecosystem composed of open-model companies and the various distributors surrounding them.

Host: Do you know what proportion of tokens processed by open models comes from models trained via reinforcement learning versus those without such training, i.e., base models?

Misha Laskin: I think every publicly available open model has undergone some form of reinforcement learning.

Host: Right, I am referring to training tailored for specific use cases or applications. In other words, custom models, as they are commonly called now.

Misha Laskin: Based on what I have heard, according to statistics cited by vendors providing dedicated inference services for open models, the vast majority are actually custom models, accounting for over 90%.

However, there is a caveat to note: today, the largest customers for specialized inference services are AI-native companies, and possibly digital-native companies. Their entire product is essentially a large-scale customization of an open model, such as Cursor, Cognition, Bridge, or Harvey. These companies build their products around heavily customized models. So, this ratio is not surprising: if inference services primarily target these types of customers, it makes sense that the models they run are mostly fine-tuned.

Nevertheless, my view differs regarding what will happen in the enterprise market. I believe that most tokens consumed by enterprises will involve customizations. I think the majority of tokens consumed by enterprises will come from models that have been adapted, rather than just raw base models.

Host: So do you think they’ll still start with the most advanced models, or closed-source models, and then migrate? Will serving frameworks become their way of customizing open models?

Misha Laskin: That’s exactly what we’re seeing. When enterprises start considering open models, they usually already have a significant workload running on closed-source models. Most of the enterprises engaging with us now are evaluating open models. I’ve rarely seen anyone jump straight to the “owning intelligence” stage. To become an owner, you typically have to go through the renting phase first.

Host: It needs to represent a large enough share of enterprise spend before you start thinking, “How do we reduce costs? This is valuable enough that we want to keep it in-house.” Then you make the switch.

Misha Laskin: Yes. For example, if you’re spending over $100 million annually on closed-source models, you start thinking, “Maybe there’s a better approach.”

I think the resurgence of on-premises infrastructure right now is a fascinating phenomenon, partly driven by compute shortages. If you’re an enterprise buying services from your preferred hyperscaler, you often find that compute capacity simply isn’t available, nor can you secure appropriate pricing. But since you’re already spending so much, you might turn to infrastructure vendors like Dell and say, “I just want to set up bare-metal infrastructure and serve it internally.” What comes next? Who helps them bridge the gap from bare metal to actual applications?

Host: And you see this as your role in the world of open models?

Misha Laskin: Exactly. We view our role as helping enterprises build solutions that successfully land, which makes us highly solution-oriented. AI-native companies can adopt open models because they build their own solutions; enterprises simply need more help in this area. The real work involves going in and assessing what is most valuable and worth doing. Many enterprises may have already built agents covering hundreds of millions of customers based on closed-source models and are eager to scale further, potentially at very high multiples. You can’t just walk in and say, “We’ll provide you with inference services.” That approach hasn’t worked in the past.

Host: The closed-source model ecosystem is fiercely competitive. Now, the open-model ecosystem seems increasingly similar. Do you agree with that assessment?

Misha Laskin: I agree.

Host: The entire field is competitive, which is typical for large markets. From your perspective and long-term positioning, what do you consider the most important competitive dimensions?

Misha Laskin: I have a simple formula for how a company ultimately wins. At its core, it depends on how much sustainable revenue you can generate, and that formula is: the density of intelligence you can provide, multiplied by the amount of compute you possess, multiplied by the trust organizations have in working with you. Ultimately, it comes down to how good you are at solving their problems.

An interesting aspect of this market is that compute is scarce, so things that weren’t previously bottlenecks have become constraints. This might be temporary, but I don’t think we should treat it as a short-term phenomenon for quite some time. Right now, even if several companies have similar products, whoever has enough compute to meet customer demand can close deals. I believe every layer of the tech stack is competing. People discuss commoditization at one layer or another, but the reality is that everything is commoditizing. Competition across the entire tech stack is extremely intense, leading each layer toward standardization and interchangeability, while compressing margins for models.

The competition between open and closed-source models definitely compresses margins; application-layer profits are squeezed, as are those in the inference layer and bare-metal infrastructure.

Host: We’re seeing many AI-native companies taking hardline negotiating stances when discussing partnerships with closed-source models, precisely because the open ecosystem exists. So, this is clearly happening.

Misha Laskin: Yes. For any company, you can understand its position through three factors: the level of intelligence density provided to customers, the amount of compute owned, and the trust gained from customers—which stems from delivering successful solutions. If these are the key factors, then as long as excellent open models exist—meaning high-density intelligence capabilities are widely accessible—there should be many successful companies. The next variable to watch is how much compute each company can organize.

We are indeed in a world where, if you excel at building intelligence—as closed-source model companies have experienced—revenue grows rapidly, allowing you to acquire more compute than others and secure a strategically advantageous position. I believe this is what it takes to win: provide excellent intelligence, hold substantial compute, and have customers who trust you.

Chinese Open Models Benefit the World; the US Still Needs to Catch Up

Host: How do you view this issue? Going back to two years ago, as mentioned earlier, many open-weight or open-source models came from the West. Mistral was developed in Europe, and Meta obviously had both European and American roots. Over the past year or two, we’ve truly seen the rise of Chinese open-weight models. There’s a perception that some of these models heavily distill capabilities from state-of-the-art models, which is part of why they’ve progressed so quickly. Now, some labs—or rather, large closed-source labs—are trying to introduce tools to prevent this distillation, or at least make it harder. How do you think Chinese open-source or open-weight models will evolve over the next year or two? Will they maintain the status quo, or will changes occur? What happens next?

Misha Laskin: First, I believe the emergence of such an excellent open-model ecosystem in China is actually very beneficial for the world. Many Western companies have been able to build more sustainable businesses precisely because of this. From that perspective, it’s a very positive development, arguably a gift. Imagine if it hadn’t happened: there might not be open models in the world at all. That would lead to a kind of “shoot the messenger” scenario: if you’re an app developer creating great products, you’d face risks because...

However, we don’t want any single domain to become a monopoly, right? For instance, if all resources and compute power are concentrated in the hands of just a few labs. If there were ten or twenty closed-source labs, you could argue that they form a competitive ecosystem; but if there’s only one or two, it becomes concerning. Similarly, when people want to own their own intelligence and build products on top of it, you wouldn’t want that source of intelligence to come from just one country. Ideally, there should be ten or twenty countries involved, but capital expenditures are so high that having at least two is already quite good. Therefore, I believe it is crucial for the West to have an ecosystem capable of competing with China. This isn’t really a “West vs. China” issue. If all the excellent open models came from another country instead of China, you would likely still want some competitive pressure. Competition itself is beneficial.

I think Chinese models will continue to remain strong. They have certain advantages and face specific challenges. One advantage is their ability to leverage the open-source ecosystem to enhance model capabilities; simultaneously, they have accumulated expertise in data acquisition. Additionally, these companies benefit from industrial support infrastructure. In the early stages, their commercial revenue was limited, but this situation is changing. Today, these companies are gradually building sustainable business operations. Their models are mostly released as open weights in overseas markets, while domestically they operate closer to a commercialized, closed-source service model. Overall, the Chinese model ecosystem is forming a genuine commercial cycle, and I believe they will continue to release competitive models.

Host: But do you think they will remain open in the future? In other words, now that there are economic drivers and business models in place, why would they continue to do so? What do you think motivates them to keep opening up their models to the West?

Misha Laskin: That’s a very interesting question. Fundamentally, enterprises, public sector entities, and sovereign clients have significant demand for Western open models for various reasons. However, there are regulatory concerns—or more accurately, uncertainties. Moreover, clients need not just the model, but also a partner who can help them run and serve the model.

The clients we engage with are mostly Fortune 500 companies. Today, the adoption rate of open models within these enterprises is actually quite low because they have reservations or concerns about Chinese models for various reasons—some rational, others perhaps not. Therefore, building such models is clearly important. If you can establish a sustainable business model that, on one hand, wins enterprise trust so they are willing to collaborate with you as a model developer, and on the other hand, enables you to secure the necessary compute resources, then there is sustained momentum for doing business around open models.

Host: I completely agree. A Western open-weight model that offers value-added services and provides inference or other services over time is extremely valuable. I also know this is the direction you are heading. I’m just curious: what are the incentives for Chinese model companies to keep their models open?

Misha Laskin: My hypothesis is that China is somewhat of a market with relatively constrained customer options, so you can build open models while still making substantial money within that market. There’s an interesting phenomenon: in the West, many excellent companies do not develop their own open models but instead provide deployment and inference services to enterprises based on existing open models. I’m not sure how prevalent this model is in China, but logically, it should exist. So it’s interesting here: open model suppliers in that country seem to effectively be intelligence service providers as well. Furthermore, I believe that consistently releasing excellent open models is geopolitically advantageous for China for several reasons. One aspect is that, as a nation, you want other countries to build their own things on top of your technology. This holds true for the US, China, and any country with such capability. In the past, there have been upgrade competitions or tensions between the US and China regarding 5G, fiber optic deployment, and the Belt and Road Initiative. Ultimately, when you enter another country and provide something inexpensive, that country subsequently becomes locked into your infrastructure. This brings commercial and geopolitical advantages to the provider country.

Host: So, I feel it’s somewhat like rare earth minerals. You control a key component that others use for various purposes, thereby gaining geopolitical influence.

Misha Laskin: Yes. One way I understand it is that open models act as a "Trojan horse" bringing accompanying infrastructure along with them. You possess an open model, but the model alone isn’t very useful. Consequently, you gradually adopt an entire software and infrastructure ecosystem from a particular country. For example, today a US ally might run Chinese models on US chips; but soon—and we’ve already seen signs of this—Chinese companies like Huawei will enter the market, providing full-stack solutions, and then deeply integrate with that country’s supply chain. So, if we view AI, or chips and data centers, as akin to railways, they are foundational infrastructure. Other countries cannot truly build such "railways" themselves; they must come to you, or to another country. As the supplier, you wield significant geopolitical influence.

Host: For instance, a specific model performs better when running inference on a certain type of chip, which happens to be provided by companies like Huawei, is that correct?

Misha Laskin: Correct. These tech stacks are undergoing real-time end-to-end optimization because, in a sense, the paradigm has been established. There may be other methods to scale intelligence in the future, but this is the first path that has clearly achieved large-scale scaling, so the entire tech stack is optimizing around it. China’s current disadvantage lies in chip performance, which isn’t yet strong enough, but it has advantages in energy, and its companies are excellent and catching up in terms of chips.

Host: And models will certainly accelerate this process, right? For example, using models to design next-generation chips.

Misha Laskin: Yes, absolutely. Additionally, there is a specific geopolitical competition targeting the US here: the fact that US companies and startups are built on foreign technology, while the rest of the world is built on your technology, is itself highly significant.

Incidentally, the US has historically done exactly this with all other critical technologies, such as the internet. Therefore, this dynamic makes sense. By the way, the US has traditionally operated this way with all major technologies, including the internet. Thus, this scenario is meaningful.

Misha Laskin: Yes, that is certainly a valid concern. However, I genuinely believe that the prevailing safety paradigm we have established today stems from a specific stance and perspective, and has become nearly dogmatic. If we set aside the debate of whether AI should be closed-source or open-source, and instead approach it from first principles—assuming we could foresee these capabilities in advance—how would you design safety from scratch?

A useful analogy is software. You can view AI as an advanced form of software; they share many similarities, both being digital general-purpose tools. Incidentally, there were also debates about whether software was dangerous, particularly regarding strong encryption protocols in the early 1990s. At that time, these protocols were actually restricted. There was an ongoing tug-of-war between the NSA and civil liberties advocates over whether to keep them closed or open them up. Ultimately, the closed side suffered catastrophic failures: systems designed by a small group of engineers produced unforeseen consequences and were easily compromised. Afterward, strong encryption protocols went open, which effectively gave birth to the entire cybersecurity field. So, the default logic of how the world works is that openness brings security. If we view AI as a continuation of technological evolution, this should have been our default starting point entering the AI era.

Models do share many similarities with software, but the stakes are higher. The long tail of vulnerabilities is longer, and models are black boxes, making them harder to understand than software, leading to more vulnerabilities and unexpected consequences. According to "Linus's Law," coined by Linux founder Linus Torvalds, given enough eyeballs, all bugs are shallow. I believe that if enough people participate in scrutiny, most information security and AI safety vulnerabilities will also become easier to resolve.

The reality today is that only a few hundred safety researchers within closed labs truly understand how these systems work. Even with the best intentions, they cannot cover the entire long tail of vulnerabilities or unexpected consequences in these systems. This has already manifested: looking at major cybersecurity incidents that have occurred, we see that a very powerful closed-source model produced unintended consequences, compromising another company. The only remedial action that company could take was to protect itself using open models. This is the fact that has already happened in the world we live in.

Moderator: By the way, what exactly do you mean by "safety"? I feel that people often conflate different concepts of safety, and when different people talk about safety, they might be referring to three or four different things. One type is security against cyberattacks, such as using AI to breach systems or perform other malicious acts; another is safety regarding biological weapons or terrorism, risks which I think are often exaggerated in the short term; and yet another is the existential threat AI poses to humanity. People often mix these issues together, but they can actually be discussed separately. So, when you talk about safety, are you addressing all three categories simultaneously? Also, I don't necessarily agree that all these risks will materialize within any realistically foreseeable timeframe. I just want to understand your perspective on openness and the benefits these approaches can bring—specifically, which problems they actually cover.

Misha Laskin: I think AI safety risks exist on a continuous spectrum, ranging from problems that are already real to those that are theoretically possible. I acknowledge that AI development sometimes feels like science fiction: things that were purely theoretical last year become reality this year, such as the capabilities of these models in cyberattacks. Therefore, we cannot dismiss theoretical risks, but we must distinguish between problems that have already occurred and those we are fairly confident will emerge within the next six months.

Moderator: Right. For example, similar concerns in the past included whether the atmosphere would ignite upon the first detonation of a nuclear weapon. That was a theoretical fear that ultimately did not happen, but a small subset of physicists did discuss it extensively. I also remember that when people talked about nanotechnology, they often mentioned "gray goo": you accidentally release a nanobot, and it suddenly consumes the entire world. This was discussed back in the 1990s. At that time, lab researchers were still working on microfluidics and similar technologies.

Misha Laskin: The difficulty with AI is that it receives so much attention, has permeated everyday culture, and is easily anthropomorphized. From a public perception standpoint, attention to various safety issues is not evenly distributed but is clearly concentrated on apocalyptic scenarios. In reality, the peak of attention should fall on realistic problems that we can actually address, tapering off gradually toward more theoretical scenarios. It does not help the discussion for corporate leaders to say, "Theoretically, there is a 10% probability that we all die," regardless of the angle. So, as a scientist, I still prefer to think in terms of distributions, focusing on reality. Looking ahead six months, as systems become stronger, I am naturally concerned that their behavior may deviate from expectations, leading to unintended consequences.

But we must start from a certain premise: these are tools built through engineering design. I recognize that some companies or researchers discuss issues like model consciousness or emotions. Perhaps they are right, but these concepts are inherently difficult to define, to the point where we aren't sure what they specifically are. So, I view them as intelligent tools and assess risks based on reality. I think theoretical apocalyptic scenarios make for good dinner conversation, but when it comes to alignment—which is a huge part of the safety problem—the real question is: how do we make these systems behave as intended? The reality is that, to date, actual progress in alignment research has been quite mundane, far less exciting than those grand visions sound. There is no magic alignment equation; it’s more like whack-a-mole: find a problem, fix a problem.

If you recall the early models before ChatGPT, some models had only undergone pre-training and would output harmful content after release. At the time, there was a claim: "These models output harmful content, and it can’t be solved because there are too many ways they generate harmful content." But later, this was addressed.

Host: I think there's an important philosophical divergence here. It sounds like you're on one side of it, but I want you to state your position explicitly. Even if these are just very intelligent tools, and even if you believe we can manage their unintended consequences with the strength of the entire ecosystem, there will still be people who say: We don't want individuals or companies having access to such powerful tools.

Misha Laskin: The premise of that argument is: We can have powerful tools, but—

Host: Others cannot, because we are actually better at managing them? That’s the underlying assumption, right? To be fair, I’m just paraphrasing; I don’t necessarily agree with this view. Their concern is that any individual or company possessing these tools could cause severe harm, yet they believe they themselves should retain access to them.

Misha Laskin: This also touches on issues similar to those surrounding weapons and different capability tiers. The US government sells fighter jets to other countries. However, generally speaking, taking a model—especially an open model that has already undergone safety training—and investing the compute, effort, and work required to make it highly dangerous is not easy.

I’m not saying it’s impossible. But consider cybersecurity again. Highly skilled hackers can indeed cause significant damage, and they have done so in the past. At the same time, we have an ecosystem where defensive capabilities typically outweigh offensive ones. If we take this as our default starting point—that this is how software protection works, that the internet contains both benevolent and malicious actors—then ultimately, the internet resembles an ecosystem with abundant white blood cells. Empowering more people with defensive capabilities actually helps resist attacks. This is true. Looking at real-world scenarios, network defense and network offense are difficult to separate. Therefore, when you remove network attack capabilities, you simultaneously remove network defense capabilities. As a result, those willing to participate in defense lose their ability to do so.

Recent events have illustrated this, such as the incident involving OpenAI and Hugging Face, where they eventually switched to using open-source models.

Host: Because existing frontier labs impose guardrails on their models, preventing them from being used effectively due to the restrictions placed on the models.

Misha Laskin: Yes. I think these arguments often drift toward extremely absolute positions, as if to say: These things are too powerful, and no one except us should ever use them. But reality never works that way. Apart from rare exceptions where there is little positive utility, it is hard to find cases where such absolutist views have truly worked.

How Researchers' Work Will Change After Accelerated Scientific Discovery

Host: Looking ahead two to five years in the AI landscape, what are you most excited about? It is admittedly difficult to predict, but following the trajectory of technological development and widespread adoption, what positive changes do you anticipate?

Misha Laskin: Many positive things are happening, and they are already underway. For instance, I am personally very excited about scientific progress. After moving to the US, I started engaging with science and became fascinated by physics. Since then, science has been my lifelong pursuit. For the past few years, I have tested language models with the same question: Submit my PhD thesis to it. Of course, finding a suitable problem for PhD research constitutes a large part of the work itself. But executing the routine calculations and research also takes a long time. Overall, from identifying the right problem to completing the solution, my doctoral research took several years.

Over the past few years, I have repeatedly fed prompts related to my thesis into language models. Two years ago, it could only chat and accomplish nothing. A year ago, it began providing undergraduate-level answers—the kind I might have given if I encountered this homework problem during my undergrad studies. Six months ago, it was solving the problem at a genuine PhD level, with correct answers. Now, when I try again, it even provides interesting new information I hadn’t considered back then. As long as you ask the right questions, you can input them into the chat interface, let it perform all the calculations, and receive genuinely interesting results.

Host: That is incredibly exciting.

Misha Laskin: Exactly. This means the iteration speed has changed. Why does it need to take years? What is the significance of a PhD project that takes years to complete? You can now finish a PhD study in a week.

Host: This is somewhat reminiscent of recent news from OpenAI, likely from the past day or two: Over the last few weeks, OpenAI proved various mathematical theorems. What is happening now is truly remarkable.

Misha Laskin: Yes, the changes these developments bring to science are absolutely stunning. I am particularly eager to see these advances benefit not only theoretical sciences conducted "on a blackboard," but also fields like life sciences, materials science, and chemistry, which require experiments in the real world.

I believe there are many areas where AI can interface with real-world experiments, establishing feedback loops with proprietary data. It will be slower—much slower than problems solved via blackboard reasoning—but still surprisingly faster than before. I also think we have consistently underestimated how much AI has already transformed software engineering and how much more we can build today. In many respects, AI is already a remarkable, highly positive tool, even for blue-collar jobs. A few weeks ago, I visited Stargate, and what impressed me most was the number of cars in the parking lot. There were a huge number of people working there. These data center projects create tens of thousands of high-paying jobs. In my view, current data centers resemble factories of the past.

The 20th century had railways and factories. Perhaps a single shoe factory could foster a vibrant economy in a small town. At least, when I visit sites under construction, I find that while people have concerns about data centers, seeing them firsthand reveals that they indeed create many job opportunities for local communities and generate substantial tax revenue. Of course, certain aspects need to be handled properly. They produce noise, so soundproofing is necessary, or they should be built away from residential areas. But one thing is certain: For local economies, the impact is clear.

When it comes to research planning, you need to strike a balance between two types of activities: carefully executing what is already known to work or making lower-risk attempts, versus betting on higher-risk directions. You should first exhaust the high-impact initiatives that are proven effective, and then gradually expand your risk tolerance. The specific approach involves using small-scale experiments to study scaling behavior before assessing the risks. Interestingly, some phenomena simply do not emerge at smaller scales. Therefore, the decision to either invest more compute to see what happens next or pivot to the next idea has both scientific and experiential/judgmental components.

Regarding models participating in the research loop, I believe they have been significantly helpful for researchers. Work that previously took one person a long time can now be completed much faster. However, these models’ capabilities remain uneven, so you need to judge which tasks are suitable for forming a continuous feedback and iterative loop with the model, and which stages still rely on human creativity. Fortunately, after using them for a while, researchers gradually understand the boundaries of the models’ capabilities, knowing which tasks can be delegated for autonomous trial-and-error improvement and which still require their own involvement. This judgment is not difficult to develop; once you identify tasks suitable for autonomous iteration by the model, you can confidently let it run continuously.

They are particularly helpful in hyperparameter search and various granular infrastructure tasks. I genuinely find these tools exciting because they allow scientists to fully leverage their intuition. It’s somewhat like having a colleague who moves very fast, is highly motivated, and is willing to listen to your input—provided it is properly aligned.

So, from my perspective, researchers have simply gained more capability, allowing them to advance work much faster than before. This explains why, when relying entirely on manual effort, the annual rate of improvement was roughly 7x, whereas now we can achieve several times that speed on top of this baseline—for example, 4–5x faster. For researchers, having such tools is indeed exciting. Of course, this raises the question: At what point will the research loop no longer require researchers?

Host: I was about to ask: If today it takes 100 people, will it take only 20 people to complete the same work next year?

Misha Laskin: For the same work, yes. But I think the scope of what you can aim to do continues to expand. So, we always feel understaffed rather than overstaffed. You still need to consider the ratio of personnel to compute; headcount cannot increase indefinitely. I don’t think 3,000 researchers would help much, but hundreds of researchers would be extremely, extremely helpful.

Host: Right. I’ve heard an argument that considers how much compute needs to be allocated per person, especially as you mentioned, if you want to scale certain experiments over time, plus the fact that contributions from different individuals often follow a distribution pattern. Ultimately, this might lead to a scenario where, in any field, a small number of people contribute the most ideas, right? Physics, biology, whatever the domain—it’s the same everywhere. Consequently, there tends to be a preference for allocating every increment of new compute to the most productive subset of people. Thus, over time, the number of researchers might remain constant or even decrease. If you push this argument to its extreme, that’s roughly the outcome.

Host: I’m not saying it’s necessarily correct, just that I’ve heard this viewpoint.

Misha Laskin: That argument does hold water. In fact, these projects have never required an absurdly large number of researchers. Projects like AlphaGo likely involved around ten people at the time. Now, larger projects like ours might involve one hundred or a few hundred people. So, it has never been about needing a massive headcount. Whether it shrinks back down to ten people eventually is hard to say. I estimate it might stabilize around one hundred, because there is simply so much to do. Additionally, applied research will create many job opportunities, essentially taking these technologies and solving real-world problems.

Host: I think people often overlook this when discussing engineering and research: these roles will diffuse into broader sectors of society, not just building language models. As you said, there are many other types of models to build and many closed-loop systems to construct. Even regarding engineers and the potential for displacement, current evidence suggests that many companies need more engineers, not fewer. Meanwhile, engineers with a certain level of expertise will move into more organizations, actually deploying these technologies—organizations that previously could not access talent of this caliber, let alone in large numbers. So, I really feel that when people think about economic implications, they often miss this point. They focus narrowly on a few big tech companies instead of asking: "What is the GDP of the entire economy? How does it change as more people bring this technology to various places?"

Misha Laskin: I think you’re right. Over the years, the number of researchers required to complete these kinds of projects hasn’t changed dramatically. It has consistently been: one major project, roughly one hundred people. I don’t think a large project requires thousands of people. However, if targeting different product forms, you might need quite a few. Interestingly, if you look closely at why even hundreds of people are needed, a significant portion of the reason is that you need different teams to evaluate different tasks and collect different data. Models possessing certain capabilities doesn’t happen by accident. There will be groups of five to ten people dedicated to tackling a specific capability; just within programming alone, there are five to ten distinct capabilities. These efforts are aimed at giving the model general-purpose abilities.

But when you want to apply the model to solve real-world problems, basically every real-world capability requires a dedicated team, and every enterprise has numerous such needs. Therefore, I believe this will create a new role: Forward-Deployed Engineer (FDE). These engineers lean more towards scientific methodology and evaluation, possessing intuition about the model and its operational framework—which is essentially what evaluation researchers do, right? The skills they need are actually similar to those of researchers building core models, just applied differently.