Upstage, an AI startup, has developed a generative AI model that outperforms OpenAI’s chatbot GPT-3.5. According to the evaluation on Hugging Face’s Open LLM Leaderboard, Upstage’s AI model, developed using Meta’s latest Language Model LLM Rama2, achieved a score of 72.3, securing the top spot. This score surpasses the performance of OpenAI’s GPT-3.5 version, which scored 71.9 in the same evaluation.
Hugging Face’s Open LLM Leaderboard is considered a benchmark for evaluating the performance of open-source generative AI models. It evaluates the performance of over 500 open models worldwide based on four metrics: inference and common-sense abilities, language understanding, comprehensive abilities, and hallucination prevention.
Previously, Upstage’s other AI model, released through Hugging Face, surpassed Meta’s Rama2 (70B model) by scoring an average of 67 and became the top-ranked domestic LLM model for the first time.
Upstage recently released a new model based on the latest Rama2 and with more data, maintaining its global top position. This new Upstage AI model also surpassed Stable Belluga2 model (71.4 score) from MobilityAI.
To develop this model, Upstage dedicated resources that have won competitions like the “Kaggle AI Olympics” and international conference paper awards. They also built the first Korean Natural Language Understanding (NLU) evaluation dataset called “KLUE” and achieved four victories in different categories at the ICDAR OCR World Championships.
In addition to their AI model, Upstage operates the AI chatbot service “AskUp,” which has gained 1.3 million users. Upstage plans to enter the “Private AI” market with its commercialized AI models. Private AI specializes in training models using only internal company data to prevent information leakage and the generation of incorrect information, addressing security concerns.
Kim Sung-hoon, CEO of Upstage, expressed his delight at the superior performance of Upstage’s generative AI model compared to GPT-3.5, stating that Upstage will strengthen its dominance in the domestic and international Private AI market with its overwhelming technological capabilities.