COMPANY NEWS: Fastly, a leader in global cloud platforms, today announced the launch of Fastly AI Accelerator, the company’s first AI solution designed to create a better experience for developers by helping to improve performance and reduce of costs when using similar requirements for large language modeling (LLM) applications.
“AI technologies in general and large model languages in particular are aggressively reshaping the tech industry and the way millions around the world – including developers – work every day,” said RedMonk Principal Analyst Steven O’Grady. “There’s a lot of focus on the bigger models. However, developers and enterprises are turning in large numbers to medium and smaller models. Whether to reduce costs, shorten training cycles or run on more limited hardware profiles, they are an increasingly important option.”
Fastly AI Accelerator is designed to reduce API calls and costs with intelligent semantic memory. Built on Fastly’s Edge Cloud platform and leveraging leading in-memory technology, AI Accelerator uses a specialized API gateway to dramatically improve performance for applications using popular LLMs, starting with ChatGPT and extending support for additional models included.
Popular AI applications can process hundreds of thousands of API calls or queries every day. Many of the questions that users ask may be very similar and may have been asked before. Without semantic caching, each call to LLM requires going back to the provider for the information, potentially increasing costs and latency. However, Fastly AI Accelerator’s semantic memory provides a cached answer to repeated queries directly from Fastly’s high-performance edge platform, rather than going back to the AI provider, helping to deliver a better experience by improve performance while reducing costs.
|
“At Fastly, we’re always listening to developers, to understand what they’re excited about and what their biggest pain points are,” said Fastly’s vice president of developer experience, Anil Dash. “The Fastly AI Accelerator gives developers exactly what they want, making their favorite LLMs experience much faster and more efficient, so they can focus on what makes their app or site tick unique and what keeps their users happy.”
When using the Fastly AI Accelerator, developers only need to update their app to use a new API endpoint, which typically only requires changing a single line of code. Fastly AI Accelerator will then transparently implement semantic storage for OpenAI-compliant APIs. This approach goes beyond traditional caching, as Fastly AI Accelerator is able to understand the context of requests and queries and will send a similar response if two or more requests are the same.
To help developers build faster, safer, and more engaging experiences, Fastly is now making it even easier for developers to try Fastly with an enhanced free account tier that helps coders set up a new site , create a new application or launch a new service in just a few minutes. Free tier accounts also include access to Fastly’s Content Delivery Network (CDN), generous memory and storage allocations, uncovered redirects, page rules, and regular expressions. Plus, the free Fastly tier includes security features like TLS and always-on DDoS mitigation, monitoring tools, and more.
To learn more about the Fastly AI Accelerator and other Fastly innovations visit fastly.com/ai. Developers looking to try Fastly for free can start here.
About Fastly
Fastly’s powerful and programmable edge-to-edge cloud platform helps the world’s best brands deliver online experiences that are fast, secure and engaging through cutting-edge compute, delivery, security and observability offerings that improve performance site, increase security and power innovation on a global scale. Compared to other providers, Fastly’s powerful, high-performance and modern platform architecture empowers developers to deliver secure websites and applications with fast time-to-market and demonstrated industry-leading cost savings. Organizations around the world trust Fastly to help them improve the web experience, including Reddit, Wendy’s, Stripe, Neiman Marcus, Universal Music Group, SeatGeek and Advance Publications. In Australia and New Zealand, the company works with organizations such as Foxtel, Linktree, Miter 10, Optus, Radio New Zealand, Seven Network, Trade Me and Vodafone NZ. Learn more about Fastly at https://www.fastly.com and follow us @fastly.
#iTWire #Fastly #helps #developers #build #internet #accelerator
Image Source : itwire.com
