Connect with us

Technology

Groq® LPU™ Inference Engine Leads in First Independent LLM Benchmark

Published

on

ArtificialAnalysis.ai Adjusts Chart Axes to Accommodate Groq Performance Levels

MOUNTAIN VIEW, Calif., Feb. 13, 2024 /PRNewswire/ — Groq®, a generative AI solutions company, is the clear winner in the latest large language model (LLM) benchmark by ArtificialAnalysis.ai, besting eight top cloud providers in key performance indicators including Latency vs. Throughput, Throughput over Time, Total Response Time, and Throughput Variance. The Groq LPU™ Inference Engine performed so well with a leading open-source LLM from Meta AI, Llama 2 70b, that axes had to be extended to plot Groq on the Latency vs. Throughput chart. Groq participated in its first public LLM benchmark in January 2024 with competition-crushing results.

“Groq represents a step change in available speed, enabling new use cases for LLMs.” – ArtificialAnalysis.ai

“ArtificialAnalysis.ai has independently benchmarked Groq and its Llama 2 Chat (70B) API as achieving throughput of 241 tokens per second, more than double the speed of other hosting providers,” said ArtificialAnalysis.ai Co-creator Micah Hill-Smith. “Groq represents a step change in available speed, enabling new use cases for large language models.”

Groq has run several internal benchmarks, reaching 300 tokens per second consistently, setting a new speed standard for AI solutions that has yet to be achieved by legacy solutions and incumbent providers. ArtificialAnalysis.ai benchmarks confirm Groq superiority over other providers, especially regarding throughput at 241 tokens per second and total time to receive 100 output tokens at 0.8 seconds according to the benchmark techniques of input prompt size and output prompt size. For more benchmark details please visit https://groq.link/aabenchmark.

“Groq exists to eliminate the ‘haves and have-nots’ and to help everyone in the AI community thrive,” said Groq CEO and founder Jonathan Ross. “Inference is critical to achieving that goal because speed is what turns developers’ ideas into business solutions and life-changing applications. It is incredibly rewarding to have a third party validate that the LPU Inference Engine is the fastest option for running Large Language Models and we are grateful to the folks at ArtificialAnalysis.ai for recognizing Groq as a real contender among AI accelerators.”

ArtificialAnalysis.ai benchmarks are conducted independently and are ‘live’ in that they are updated every three hours (eight times per day). Prompts are unique, around 100 tokens in length, and generate ~200 output tokens. This is designed to reflect real-world usage and measures changes to throughput (tokens per second) and latency (time to first token) over time. Benchmarks are also present on ArtificialAnalyis.ai with longer prompts to reflect retrieval augmented generation (RAG) use cases.

The LPU Inference Engine is available through the Groq API. For access, please complete the request form at https://groq.link/contact.

About Groq

Groq® is a generative AI solutions company and the creator of the LPU™ Inference Engine, the fastest language processing accelerator on the market. It is architected from the ground up to achieve low latency, energy-efficient, and repeatable inference performance at scale. Customers rely on the LPU Inference Engine as an end-to-end solution for running Large Language Models (LLMs) and other generative AI applications at 10x the speed. The LPU Inference Engine is available via the GroqCloud, an API that enables customers to purchase Tokens-as-a-Service for experimentation and production-ready applications. Jonathan Ross, inventor of the Google Tensor Processing Unit (TPU), founded Groq to preserve human agency while building the AI economy. Experience Groq speed for yourself at https://groq.com/.

Media Contact for Groq
Allyson Scott
PR-media@Groq.com

 

 

View original content to download multimedia:https://www.prnewswire.com/news-releases/groq-lpu-inference-engine-leads-in-first-independent-llm-benchmark-302060263.html

SOURCE Groq

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Technology

Stay Better in China: Civilization is Colorful for Communication

Published

on

By

NANCHANG, China, Oct. 18, 2024 /PRNewswire/ — A report from Jiangxi International Communication Center(JXICC):

 

Moses, an international student from East China University of Technology, has stayed in China for four years. He experienced the speed of China and the colorfulness of Chinese civilization.

Themed on Chinese-style modernization, Moses shares his wonderful lives and touching stories in China from various aspects such as cultural construction, industrial development, ecological protection, and rural revitalization. He is eager to bring Chinese-style modernization experience back home for contributing to the prosperity and development of their countries. He also expects China to play a more active role on the global stage and contribute more wisdom and strength to world peace and development. He believes that these beautiful visions will certainly become reality as China continues to develop and make progress.

https://www.youtube.com/watch?v=tBjk0tXlHDE

View original content to download multimedia:https://www.prnewswire.com/news-releases/stay-better-in-china-civilization-is-colorful-for-communication-302277208.html

SOURCE Jiangxi International Communication Center(JXICC)

Continue Reading

Technology

The AI solutions for Japanese businesses make a global splash with the NVIDIA promotion campaign

Published

on

By

The tech giant NVIDIA highly appreciates the AI solutions developed by FPT Smart Cloud in elevating customer experience and optimizing workforce competency.

HANOI, Vietnam, Oct. 18, 2024 /PRNewswire/ — In April 2024, NVIDIA joined forces with FPT Corporation in the global initiative of AI and Cloud advancement. After 6 months, the AI solutions of FPT Smart Cloud – a subsidiary of FPT Corporation, have positioned themselves amidst the world’s 52 outstanding success stories featured by NVIDIA. FPT is currently offering more than 15 state-of-the-art AI solutions built on the natural language model for Japanese.

Enabling the intelligent call center 

In the digital age, AI virtual assistants are integrated into the contact center to automate simple to sophisticated tasks.

According to NVIDIA, FPT Smart Cloud has enabled seamless communication through the development of FPT AI Engage – the virtual agent for call center. The virtual agent can interact naturally and automate inbound calls, outbound calls, and voice-based call transfer (IVR). Operating on NVIDIA’s most advanced infrastructure, the AI vendor has been able to accelerate the speech synthesis model by 4 times and increase virtual agent efficiency.

The AI capabilities of FPT Smart Cloud have attracted a wide range of banking and financial institutions, particularly Home Credit Vietnam. The consumer finance company has been applying FPT AI Egnage since 2019. In the first year of operations, the virtual assistant can automatically process up to 12 million calls per month, allowing Home Credit to save 50% in operating expenses and attain a success call rate of 98%.

Building the AI-powered workforce of the future 

With the support of NVIDIA A100 and H100 GPUs, FPT Smart Cloud has developed FPT AI Mentor, fostering the next-generation employee training through automation and personalization. FPT AI Mentor applies the large language model (LLM) developed on NVIDIA DGX H100 and supported by NVIDIA NGC and PyTorch to generate and customize learning content based on the business knowledge base.

Long Chau, a pharmaceutical chain with 1,700 stores, has successfully integrated FPT AI Mentor into the daily training process for pharmacists, increasing employee knowledge quality by up to 55% while reducing 30% of resources compared to the traditional method.

NVIDIA further appraises the excellent efforts of FPT Smart Cloud in leveraging AI technologies to transform traditional call center operations and business training processes. In the past three years, FPT Smart Cloud invested on R&D initiatives for Indonesian and Japanese natural language processing technologies. In 2023, FPT Smart Cloud with the bilateral collaboration with Home Credit Indonesia, marking a crucial milestone as the first made-in-Vietnam AI solution to successfully attract and win over large enterprises in the global landscape. For the year 2024, the vendor aims further to conquer international markets, including Indonesia and Japan.

The original blog about FPT Smart Cloud solutions on the NVIDIA website can be found here: https://www.nvidia.com/en-us/case-studies/fpt-smart-cloud-levels-up-customer-service-operations/

View original content to download multimedia:https://www.prnewswire.com/apac/news-releases/the-ai-solutions-for-japanese-businesses-make-a-global-splash-with-the-nvidia-promotion-campaign-302280179.html

SOURCE FPT Smart Cloud

Continue Reading

Technology

Husqvarna Group appoints Maha Elkharbotly as President of the Gardena Division

Published

on

By

STOCKHOLM, Oct. 18, 2024 /PRNewswire/ — Maha Elkharbotly has been appointed President of the Gardena Division and will also be a member of Husqvarna Group’s executive management team.

Maha Elkharbotly holds an MBA in Marketing from the University of Illinois and is currently President at I-Health, a wholly owned subsidiary of DSM Firmenich. Maha has previously held multiple executive positions at DSM Firmenich, LIXIL, Grohe and Whirlpool.  

“I am very pleased to welcome Maha to Husqvarna Group. Her broad experience and strong knowledge in sales, marketing, channel management and leadership will be valuable assets in the continued journey of Gardena. Maha brings vast experience from building brands as well as driving international expansion, innovation and business transformation. This expertise will be central to continue building Gardena’s position as a global leader in watering, as well as advancing smart garden systems including robotic mowers for passionate gardeners”, says Pavel Hajman, CEO of Husqvarna Group.  

Maha Elkharbotly will be based in Ulm, Germany and assume her new position within the Gardena Division on January 1, 2025.

For additional information, please contact:
Media
Henrik Sjöström, Head of External Communications
+46 727 15 77 85
press@husqvarnagroup.com

Investors
Johan Andersson, Vice President Investor Relations
+46 702 100 451
ir@husqvarnagroup.com

This information was brought to you by Cision http://news.cision.com.

https://news.cision.com/husqvarna-group/r/husqvarna-group-appoints-maha-elkharbotly-as-president-of-the-gardena-division,c4053079

The following files are available for download:

https://mb.cision.com/Main/996/4053079/3061209.pdf

Husqvarna Group appoints Maha Elkharbotly as President of the Gardena Division

https://news.cision.com/husqvarna-group/i/maha-elkharbotly-2,c3343479

Maha Elkharbotly 2

 

View original content:https://www.prnewswire.com/news-releases/husqvarna-group-appoints-maha-elkharbotly-as-president-of-the-gardena-division-302280185.html

SOURCE Husqvarna Group

Continue Reading

Trending