Connect with us

Technology

Tetrate and Bloomberg collaborate on community-led open standard for AI Gateways built on CNCF’s Envoy Gateway project

Published

3 hours ago

on

Joint effort will expand capabilities of the Envoy Gateway project, one of the Kubernetes Gateway API implementations.

SAN FRANCISCO, Oct. 3, 2024 /PRNewswire-PRWeb/ — Driven by the surge in applications built to utilize large language models (LLMs), the open source AI gateway space is heating up. In response, engineers from Bloomberg and Tetrate have partnered to develop an innovative, community-led set of core AI gateway features for enterprise AI integration. This effort will expand the capabilities of the CNCF’s Envoy Gateway project, one of the Kubernetes Gateway API implementations.

As an ‘open source first’ company, Bloomberg believes in the power and collaborative nature of the open source community to develop web scale solutions, and that important difference makes this project a valuable alternative to other ongoing efforts.

As the emerging open standard for handling Kubernetes ingress traffic, Envoy Gateway is designed for at-scale operation and is extensible, making it a solid choice to underpin this new set of features, as well as future innovation in the AI API gateway space. Furthermore, Envoy Gateway is a community-led open source project with no commercially licensed features, and where community members drive decisions about upstream feature development.

This sets it apart from both existing vendor-led open source AI gateway offerings and fully proprietary, commercial AI gateway solutions that have sought to address this problem. Both of these approaches can create greater complexity for some enterprises and hamper innovation, which is why the Envoy community is developing an option without vendor lock-in or features that can only be accessed through additional, paid enterprise licenses.

“Historically, when shared problems arise in the software industry, the open source community rallies to solve them, accelerating innovation,” said Varun Talwar, founder of Tetrate. “Our collaboration with Bloomberg and the CNCF aims to achieve precisely that: designing and delivering a community-led, fully open source AI gateway, powered by the leading contender to replace legacy models for Kubernetes ingress. It’s a solution the market is asking for, and we’re excited to be part of the team of maintainers and contributors creating it.”

AI Gateways enable organizations to integrate AI functionality into workflows and applications. They route requests to multiple AI service providers and models through a single reverse proxy layer (often referred to as a gateway). AI Gateways simplify AI integration by providing a single unified API layer with which developers interact, and can provide additional functionality, such as rate limiting, caching and observability.

The initial idea for this project arose when Dan Sun, engineering team lead for Bloomberg’s Cloud Native Compute Services – AI Inference team and co-founder/maintainer of the KServe project, came to the Envoy community and outlined his views of the problem space and a potential path forward for solving it. Tetrate, a major upstream contributor to the Envoy project, stepped forward to express interest in helping Sun and Bloomberg turn their vision for the Envoy AI Gateway API into reality.

“Bloomberg has more than 15 years of experience delivering value to our customers by incorporating artificial intelligence (AI) – in particular, machine learning and natural language processing – in enterprise applications,” said Steven Bower, engineering lead for Bloomberg’s Cloud Native Compute Services group. “When we looked to the community for someone to collaborate with to start building gateway features that accelerate AI integration in our products, we immediately identified the engineering team at Tetrate. Their people are leading contributors to Envoy Gateway, and they bring significant expertise to the project around handling cloud-native, scalable traffic. Beyond that, as an ‘open source first’ company, Bloomberg believes in the power and collaborative nature of the open source community to develop web scale solutions, and that important difference makes this project a valuable alternative to other ongoing efforts.”

Envoy Gateway and KServe can be used together to allow traffic routing to both self-hosted and vendor-hosted LLMs. In this case, the AI gateway sits on the top and routes open source LLM model traffic to self-hosted endpoints using KServe, and vendor-hosted model traffic is routed to AWS Bedrock or other, similar cloud-based services.

The first features to be included in Envoy AI Gateway will provide:

application traffic management to LLM providers with high-availability routing strategies;LLM usage monitoring and control at the application, organization and enterprise levels, to help users manage costs; anda unified interface for LLM requests through which the gateway handles back-end connectivity to various LLM providers.

The open source Envoy Gateway extensions and enhancements will offer usage control for applications that are integrated with multiple LLM providers and models, robust authorization mechanisms, and intelligent fallback options to ensure continued operation even when cloud providers are unavailable or too expensive.

This open source initiative, part of the Cloud Native Computing Foundation (CNCF), is more than just a tool; it’s a strategic response to challenges enterprises face in adopting and integrating AI in their applications at scale. By laying the groundwork for scalable AI platforms, Tetrate and Bloomberg engineers are addressing the immediate needs of today’s enterprises and setting the stage for the future of AI applications within cloud-native environments.

“The Envoy project continues to impress with its flexibility to support new and valuable use cases,” said Chris Aniszczyk, CTO of the CNCF. “Bloomberg and Tetrate have done exactly what our community is designed to do: bring people and organizations together to solve a common problem. That they’re doing it with Envoy Gateway only validates the power and extensibility of the project.”

*Learn More About the Envoy AI Gateway Project*
Join the “Enabling AI Adoption at Scale – The AI Platform with Envoy AI Gateway” webinar, an online panel discussion of Bloomberg and Tetrate engineers hosted by the CNCF on Thursday, October 17, 2024 at 1 PM EDT. Erica Hughberg of Tetrate will be joined by Dan Sun and Yuzhui Liu of Bloomberg and other contributors from Tetrate and the CNCF community for a conversation about the project and ways for others to get involved and contribute.

The session will include discussions about:

Industry trends from traditional AI to generative AIWhat to think about when enabling the use of AI in an enterprise organizationThe role of the AI Platform – what it solves and what it doesn’tWhy build an AI Gateway using Envoy and Envoy Gateway?How do I get started building my organization’s AI Platform?

Register for the online panel here.

In addition, attendees at KubeCon + CloudNativeCon North America 2024 (November 12-15) in Salt Lake City, UT, will have the opportunity to learn more about the project and how to get involved in it. A demo of the Envoy AI Gateway will also be available in the Tetrate booth (Booth Q2).

About Tetrate
Rooted in open source, Tetrate was founded to solve the application networking and security challenges created by modern computing so enterprises can innovate with speed and safety in hybrid and multi-cloud environments. As applications evolve into collections of decentralized microservices, monitoring and managing the network communications and security among those myriad services becomes challenging. This is why some of the largest financial institutions, governments and other enterprises rely on Tetrate to deliver modern application networking and security. Find out more at http://www.tetrate.io.

About Bloomberg
Bloomberg is a global leader in business and financial information, delivering trusted data, news, and insights that bring transparency, efficiency, and fairness to markets. The company helps connect influential communities across the global financial ecosystem via reliable technology solutions that enable our customers to make more informed decisions and foster better collaboration. For more information, visit Bloomberg.com/company or request a demo.

About the CNCF
Cloud native computing empowers organizations to build and run scalable applications with an open source software stack in public, private, and hybrid clouds. The Cloud Native Computing Foundation (CNCF) hosts critical components of the global technology infrastructure, including Kubernetes, Prometheus, and Envoy. CNCF brings together the industry’s top developers, end users, and vendors and runs the world’s largest open source developer conferences. Supported by more than 800 members, including the world’s largest cloud computing and software companies, as well as over 200 innovative startups, CNCF is part of the nonprofit Linux Foundation. For more information, please visit http://www.cncf.io.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page. Linux is a registered trademark of Linus Torvalds.

Media Contact

Cristin Connelly, Tetrate, 4049316752, cristin@cathey.co, www.tetrate.io 

Chaim Haas, Bloomberg, 212-617-7025, chaas30@bloomberg.net, www.bloomberg.com

View original content to download multimedia:https://www.prweb.com/releases/tetrate-and-bloomberg-collaborate-on-community-led-open-standard-for-ai-gateways-built-on-cncfs-envoy-gateway-project-302266565.html

SOURCE Tetrate

Related Topics:
Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Technology

GovExec Unveils the 2024 Leading Brands in Government Study Results

Published

57 mins ago

on

October 3, 2024

By

Annual study offers actionable insights to help companies position to win in 2025.

WASHINGTON, Oct. 3, 2024 /PRNewswire/ — GovExec, the leading information services company serving government leaders and contractors, released the 2024 Leading Brands in Government study, at an event packed with industry leaders on September 19 at the Archer Hotel in McLean.

For over a decade GovExec’s Leading Brands effort has been both a tool for government marketers AND a method by which we can highlight those federal brands that have been successful in their marketing and advertising efforts.

“Since it launched ten years ago, Leading Brands has provided the industry a standard against which to measure themselves year over year,” said Tim Hartman, CEO of GovExec. “Leading Brands provides public sector companies with actionable, proven insights to boost their market perception. The study captures the respondent priorities and perceptions of contractors in both the Federal and Defense markets, pulling together a compilation of top trends that provide a roadmap for best practices in public sector marketing.”

GovExec surveyed over 1,000 Federal and Defense decision makers about their perceptions of 58 contractors for the 2024 Leading Brands study. Respondents were asked to consider criteria including brand familiarity and favorability, as well as attributes identified as highly important to government purchasers — expertise, experience, innovation, customization, value, trustworthiness, customer service and social responsibility.

This year, the study found that on average, 83% of respondents have some level of awareness of the companies studied. This number includes individuals who just recognize the name to those who are very familiar with the company. However, of those 83%, on average only 42% are Very Familiar with the company.

“This gap indicates a need for more brand awareness marketing in the public sector overall.” said Aaron Heffron, President, Forecasting and Research at GovExec.

Leading Brands provides a hands-on tool for government contractors to improve their connections with the government community. It helps marketers (whether or not among the companies studied) answer the questions:

Do our marketing and advertising efforts make a difference?What should we do in the coming year to move the needle?How are our competitors doing? Is there an opening or an opportunity for us?In general, How are we doing?

“Insights like this offer actionable touch points where companies can enhance market perception and navigate today’s complex and evolving landscape with the right tools and resources,” said Heffron.

To learn more about purchasing the study to help position your company brand in 2025, contact Caitlyn O’Connor at coconnor@govexec.com.

View the 2024 Leading Brands launch event via on-demand here.

About GovExec:
GovExec’s data and insights set the standard for depth, accuracy, and impact for government leaders and contractors. As the market-leading information and intent-based marketing platform, for over fifty years GovExec has empowered the government ecosystem to engage and support government leaders as they work to achieve their missions across federal, defense, and state and local agencies. Our strategic sales enablement and intent-based marketing solutions accelerate revenue growth to fuel market success. The platform is powered by the largest and most sophisticated database in the public sector, GovExec’s platform reaches 3.3 million government influencers each month and provides its marketing clients with a significant competitive advantage in driving higher conversion and growth.

View original content:https://www.prnewswire.com/news-releases/govexec-unveils-the-2024-leading-brands-in-government-study-results-302267109.html

SOURCE GovExec

Continue Reading

Technology

NASA Invites Media to SpaceX’s 31st Resupply Launch to Space Station

Published

57 mins ago

on

October 3, 2024

By

WASHINGTON, Oct. 3, 2024 /PRNewswire/ — Media accreditation is open for the next launch to deliver NASA science investigations, supplies, and equipment to the International Space Station. This launch is the 31st SpaceX commercial resupply services mission to the orbital laboratory for the agency and will liftoff on the company’s Falcon 9 rocket.

NASA and SpaceX are targeting no earlier than Wednesday, Oct. 30, to launch the SpaceX Dragon spacecraft from Launch Complex 39A at NASA’s Kennedy Space Center in Florida.

Credentialing to cover prelaunch and launch activities is open to U.S. media. The application deadline for U.S. citizens is 11:59 p.m. EDT, Tuesday, Oct. 15. All accreditation requests must be submitted online at:

https://media.ksc.nasa.gov

Credentialed media will receive a confirmation email upon approval. NASA’s media accreditation policy is available online. For questions about accreditation, or to request special logistical support, email: ksc-media-accreditat@mail.nasa.gov. For other questions, please contact NASA’s Kennedy Space Center newsroom at: 321-867-2468.

Para obtener información sobre cobertura en español en el Centro Espacial Kennedy o si desea solicitor entrevistas en español, comuníquese con Antonia Jaramillo o Messod Bendayan a: antonia.jaramillobotero@nasa.gov o messod.c.bendayan@nasa.gov.

Each resupply mission to the station delivers scientific investigations in the areas of biology and biotechnology, Earth and space science, physical sciences, and technology development and demonstrations. Cargo resupply from U.S. companies ensures a national capability to deliver scientific research to the space station, significantly increasing NASA’s ability to conduct new investigations aboard humanity’s laboratory in space.

In addition to food, supplies, and equipment for the crew, Dragon will deliver several new experiments, including the Coronal Diagnostic Experiment, to examine solar wind and how it forms. Dragon also delivers Antarctic moss to observe the combined effects of cosmic radiation and microgravity on plants. Other investigations aboard include a device to test cold welding of metals in microgravity, and an investigation that studies how space impacts different materials.

Crews have occupied the space station continuously since November 2000. In that time, 280 people from 23 countries have visited the orbital outpost. The space station is a springboard to NASA’s next great leap in exploration, including future missions to the Moon under Artemis, and ultimately, human exploration of Mars.

Learn more about NASA’s commercial resupply missions at:

https://www.nasa.gov/station

View original content to download multimedia:https://www.prnewswire.com/news-releases/nasa-invites-media-to-spacexs-31st-resupply-launch-to-space-station-302267112.html

SOURCE NASA

Continue Reading

Technology

Goodnotes Elevates Productivity for Its 24M+ Users with New AI-Powered Enhancements

Published

57 mins ago

on

October 3, 2024

By

Ask Goodnotes, Math Assist, and new handwriting editing capabilities enhance how users work, learn, and create

LONDON, Oct. 4, 2024 /PRNewswire/ — Goodnotes today introduced three AI-powered upgrades that build on its legacy of enabling smarter, more flexible ways to take notes and unlock the value within them. With over 24 million monthly active users, Goodnotes is now seamlessly integrating advanced AI where it matters most: right where people are already capturing and exploring ideas, learning new concepts, and staying productive.

 

Unveiled during a livestream event today, the updates build on more than 40 new features the company has brought to Goodnotes 6 in the past year. The new Ask Goodnotes lets users ask questions and get AI-powered answers based on their personal notes, right from within the Goodnotes 6 app. New handwriting editing capabilities make adjusting and reformatting handwriting as simple as typed text, helping users stay in flow as they take notes and develop ideas. And the new Math Assist instantly recognizes handwritten equations, offering quick calculations or suggesting next steps.

“We’re constantly inspired by the sheer volume of ideas and knowledge that our users capture in their Goodnotes notebooks. Our aim with Ask Goodnotes is to give users new powers to interact with their notes, documents, and PDFs, and unlock fresh possibilities for productivity, creativity, and learning,” said Steven Chan, founder and CEO of Goodnotes. “With our new handwriting editing and math features, we focused on how our proprietary machine learning models could be leveraged behind-the-scenes to make everyday note-taking and document annotation more seamless and intuitive.”

Ask Goodnotes allows users to ask questions, get summaries, clarify concepts, and generate interactive quizzes based on their personal notes. The AI assistant draws from all the various content types users may have in their notes, including handwriting, typed text, diagrams, and imported PDFs. Tuned for safety and reliability, Ask Goodnotes’ answers include citations that link back to the source material within the notebook and specify when information is drawn from general knowledge outside users’ documents. Accessible in one tap in any notebook, Ask Goodnotes is thoughtfully integrated in user workflows, including enabling drag-and-drop of answers right into notes and providing custom suggested questions based on document content.

Powered by Goodnotes Smart Ink—the company’s advanced in-house AI handwriting technology—the new handwriting editing capabilities in Goodnotes make it effortless to reflow, align, copy-paste, and otherwise adjust handwriting. These updates build on Goodnotes’ past breakthroughs in its proprietary AI handwriting technology, including the launch of the world’s first generative AI for handwriting with features like Spellcheck and Word Complete back in August 2023. Whether correcting meeting notes on the fly, making space for even more ideas in a mindmap, or reorganizing study materials for better learning, users can now edit and adjust their handwriting as easily as typed text.

Developed in response to the popularity of Goodnotes 6 features such as math handwriting recognition and interactive exam practice features, Math Assist provides instant math solutions and suggestions right within notebooks. Users can get a hand with math simply by writing a line of math followed by an equal sign, enabling quick on-the-fly calculations. For those building their confidence in advanced math including trigonometry and calculus, Math Assist offers suggested next steps for math expressions, system of equations, functions, and more.

These new features are all available as part of Goodnotes’ existing plans: Math Assist and the new handwriting editing features for all Goodnotes users, and Ask Goodnotes at no additional cost for all users with a paid plan (excepting regions with restrictions). They join a growing slate of AI-powered Goodnotes features, including Spellcheck and Word Complete for handwriting, handwriting recognition, math conversion, scribble to erase, circle to select, live audio transcription, and more.

Recognized as Apple’s 2022 iPad App of the Year and one of the “Best iPad apps to boost productivity and make your life easier” by TechCrunch, Goodnotes continues to lead the way in delivering innovative, user-centric features for productivity and creativity.

About Goodnotes

Goodnotes is the leading AI digital paper used by millions of users worldwide. Launched in 2011, Goodnotes started as an improvement to physical paper notes—introducing the ability to take handwritten digital notes, search handwritten text, and organize everything into a digital library. Today, Goodnotes is pioneering generative AI for digital handwriting in the productivity space. Goodnotes was named Apple’s 2022 iPad App of the Year.

 

View original content to download multimedia:https://www.prnewswire.com/apac/news-releases/goodnotes-elevates-productivity-for-its-24m-users-with-new-ai-powered-enhancements-302267113.html

SOURCE Goodnotes Limited

Continue Reading

Trending