Why Rebellions’ strategy against Nvidia is not so outrageous

Industry

Published: 07 May. 2024, 08:37

Why Rebellions’ strategy against Nvidia is not so outrageous

Rebellions CEO Park Sung-hyun poses after an interview at the AI chip startup's headquarters in Bundang, Gyeonggi in March. [PARK SANG-MOON]

[INTERVIEW]

In the Nvidia-dominated world of AI chips, a five-year-old startup from Korea, Rebellions, has some outrageous ideas to win the game.

Leveraging the benefits of a fast-follower, it is eying a shortcut to the top of AI inference, one of two major pillars in the AI ecosystem along with training. When that time comes, the idea of going shoulder-to-shoulder with Nvidia, in inference at least, won't sound so outrageous.

"In the past, before ChatGPT's emergence and before Nvidia's market dominance, it was like racing in the dark," says Park Sung-hyun, co-founder and CEO of Rebellions, an AI chip company based in Bundang, Gyeonggi, specializing in neural processing units (NPU) for AI inference, in an interview with the Korea JoongAng Daily.

Park, a native of the coastal city of Ulsan, did his master's and doctorate's degrees at the Massachusetts Institute of Technology before building his career as a chip expert at Intel, SpaceX and Morgan Stanley.

"But now the market has become clear with large language models (LLM) becoming the mainstream. And Nvidia is now presenting us with a road map. That guideline is a big help to small followers like us to quickly catch up in the market."

Park is putting up a strong fight against a coalition of big players like Nvidia, SK hynix and TSMC, while Rebellions relies solely on Samsung Electronics.

For small players like his company, however, he contends that having one partner that is able to handle everything is better as it saves resources.

"It's a turnkey with Samsung since it handles memory, manufacturing, and packaging," he says. "It is only one channel that we have to manage."

Park is confident that in AI inference, which prioritizes speed and cost efficiency over heavy computational skills in training, Rebellions can do better than others with its talented engineers and solid partners such as KT and Samsung Electronics.

His confidence is backed by hard evidence.

It recently closed on 165 billion won ($121 million) in Series B funding from global investors including Korea's KT as well as France's Korelya Capital and Japan's DG Daiwa Ventures.

The investment led its valuation to reach 880 billion won, leaving it only inches short of becoming a unicorn firm.

It was the only Korean company to make the list of the top 100 AI companies compiled by CB Insights that was released in April, joining OpenAI, Hugging Face, Anthropic and Groq among others.

This year is expected to be a major milestone for Rebellions because its ATOM NPU started mass production in March and a more premium REBEL NPU is slated for a tape-out in September with a plan for mass production next year in Pyeongtaek and Taylor, Texas.

A handful of other AI inference chips will flock to the market this year, two years on from the initial hype surrounding ChatGPT, and the real game to prove that Rebellions' silicon stands out among others will begin.

"We are going for the major league," Park says.

Below are the edited excerpts from the interview.

In inference, can you give details on how Rebellions' chip is better than others?

If you are to run Meta's LLaMA-65B model, you need two of Nvidia's H100 GPU, which has a memory capacity of 80GB. Our Rebel chip, which is set for a tape-out this September, has a memory capacity of 144GB. You only need one of our chips to run that language model compared to two of Nvidia's.

What about the cost then?

We have a target of offering an inference performance threefold that of Nvidia's at one-third of their price.

Can you elaborate more on your fast-follower strategy?

Before the advent of ChatGPT, AI was for only a few people. AI chip companies including us, in the middle of working on the ATOM chip for finance and small data centers, ended up having the product-market-fit all messed up because we didn't know that AI services like ChatGPT, which are for the masses and require a huge amount of data processing, will become mainstream. Nvidia also did not expect it, but due to its expansive portfolio, it was able to respond to the market well. Before, it was a game that companies with big portfolios were destined to win. Now, with the market narrowed down, it is a game of who can make a product that perfectly fits the market. And we are saying we can make it, and we can make it better than others.

Did you have to start the design and business strategy all over again with the advent of ChatGPT?

Actually it was an extension of what we were doing. What we prioritized was latency when we were focusing on chips for the finance market. In inference for LLMs, latency, an indicator of how fast the model coughs up its first response, which we call the first token, is crucial as well. So it was a natural flow.

You closed Series B funding early this year with 165 billion won, which is a notable feat considering not-so-favorable market conditions. Now your accumulated investment adds up to 277 billion won. What was the process like?

We were able to persuade investors that Rebellions is the top player for AI chips in Korea with our talent pool. One of our founding members is Dr. Oh Jin-wook, our CTO who is from IBM and has a global reputation. In fact, although I am sitting here doing the interview, when abroad, foreign investors call us Jin-wook's team. Another founding member is CPO Dr. Kim Hyo-eun, who was already respected in this technology scene, having served as CPO at Lunit. Kim's reputation goes as far as being able to start a company of his own now and still being able to raise funds. In the AI chip scene, it is impossible to make an investment decision after the silicon is out in the market because it requires a tremendous amount of money and also time. Investors make decisions first from a human resource perspective, and then in a second round with the product. Our strategic partner KT also played a big part.

KT, Korea's second-biggest telecom company, invested a total of 66.5 billion won into Rebellions so far and is its strategic partner, owning at least 10 percent of the startup. Rebellions has also provided its ATOM chips to KT's data center to run its cloud services.

KT doesn't perceive AI as some cool, high-tech algorithm. It sees it as an infrastructure. They plan on building a range of data center infrastructure to enable AI. They are good at building infrastructure, having done so with 3G, 4G and 5G networks. KT is actually the biggest operator of internet data centers in Korea. When the investors carried out CDD, or commercial due diligence, the fact that a local company has a local data center was perceived well. KT's strategic investor position also served as a kind of shield because the investors would think that with such a big company behind us, Rebellions will never go out of business.

The Rebel chip, obviously your main product since it is named after the company, is scheduled for a tape-out in a few months. What has the initial response from the market been like?

It is positive. We are having meetings with Google and other Big Tech and telecom companies from Japan. We are also running a qualification test with IBM, which is known for taking a very conservative approach to such tests. This is going to take a while because they are running aging tests as well to determine the chips' durability. One of the criteria they look at closely is the product's reliability. They have a very high standard because their systems are used by the military as well as in enterprise and public finance. If we qualify at a certain level in IBM's test, it would allow us to take an easier path with other tech companies, which is why we have targeted IBM first.

You have a close partnership with Samsung Electronics. What has that been like?

Yes, we are deploying their HBM3E (the next-generation high bandwidth memory) chips on our Rebel and we are contracting its manufacturing at Samsung's Taylor, Texas factory utilizing its four-nanometer process. People may cast doubt because with foundries (contract manufacturing), TSMC is as you know No. 1. But as much as foundry is important for AI chips, memory chips are also crucial. TSMC doesn't have it. And it doesn't end there. Chiplet packaging technology that connects all the different components together is also important. And you know what? Samsung Electronics has it all. For a small startup like us, it is difficult to manage a supply chain involving different companies like HBM from SK hynix, packaging from Amcor and foundry with TSMC. In that sense, Samsung Electronics is a good partner.

Rebellions' ATOM chip, which started mass production in March [REBELLIONS]

Did the Rebel chip's early production phase with Samsung go well, such as the yield?

It is hard for us to know the yield, but the outcome matched what they told us, such as if you put in a certain amount of wafer you will get certain amount of output. Actually, the outcome was better than expected because their PDK (process design kit; a set of files delivered by chip manufacturers to the designers about the fabrication process) has been set very conservatively. They don't exaggerate it. Perhaps that's because they don't have that many clients yet. If they exaggerate the PDK, they will not be able to secure clients in a long term.

What is your plan with going public?

We are going there for sure and we have started preparations. We are going to list on the Korean market first and head to the U.S. for dual listing. We plan on running in the "K-league" first until our valuation reaches somewhere near $4 billion and then move to the major league in the U.S.

BY JIN EUN-SOO, PARK EUN-JEE [jin.eunsoo@joongang.co.kr]