The Rise of Smaller, Cheaper AI Models

TECH IN THE NEWS

Wiz - The Israeli cybersecurity startup rejected Alphabet’s $23b takeover bid, instead opts to pursue an IPO. It’s seen as a big setback for Alphabet’s hopes to compete with Microsoft and Amazon in the cloud services market.

Strike Out - Microsoft says ~8.5M of its devices were affected by the CrowdStrike-related global tech outage, ~half the world's computers.

Ethereum ETFs - The SEC has approved US Spot ETF trading for the second-largest cryptocurrency, funds may trade as early as Tuesday.

Harris On Tech? - Presumptive Democrat candidate, Kamala Harris is seen by many as soft on curbing big tech power, and tough on regulating AI, saying she rejects “the false choice” of AI regulation vs innovation.

Coalition For Secure AI - Google, Amazon, Nvidia team up to promote the development of “secure-by-design” AI, to address software supply chain security and cybersecurity risks for AI systems.

Perplexity Woes - The Condé Nast media conglomerate has sent a cease-and-desist letter to AI search startup Perplexity, accusing it of plagiarism.

#BUZZCHALLENGE
Grow Your Business with HubSpot's Free CRM

• HubSpot offers an intuitive customer relationship management platform tailored for small businesses.

• Manage leads, track sales performance, and understand your customers with ease.

• Best of all, it’s completely free, with no limits on users or data, allowing you to store and manage up to 1,000,000 contacts.

THE HOTTEST THING IN TECH
The Rise of Smaller, Cheaper AI Models

The price of performing some AI tasks just got cheaper. 20x cheaper in some cases. Competition is heating up in the Small Language Model (“SLM”) space, with Meta’s open source Llama 3.1 anticipated to drop today, and GPT-4.0 Mini wowing early users this week. Apple also just unveiled a new 7b parameter o-s model, DCLM-7B, that closes in on leading open source small models like Llama 3 and Gemma across key benchmarks.

Size Matters

The availability of smaller o-s models from players like Mistral’s Nemo, and OpenPipe, Octo AI and Together AI combine the convenience of OpenAI's API, with the cost savings and customizability of open-source models. This, as well as competition from bigger OS rivals like Meta (and soon Apple), has led OpenAI to create their own alternatives.

This is a big deal for the sector as it means AI use can more effectively be routed to the right level for each task, freeing up LLMs for advanced tasks and creating more compute efficiency for the companies and data centers. It ultimately means consumers increasingly benefit from a greater range of options and less overheads, making AI apps a more viable business model.

GPT-4o Mini: Small Model, Big Impact

A recent report surprisingly found that a majority of OpenAI's revenue actually comes from ChatGPT consumer subscriptions. However, its API is fundamentally what has driven rapid, exponential innovation in the app layer startup space. It now plans to build its own ecosystem around the framework offering options to “fine-tune” GPT-4o mini. This is akin to Llama letting customers train self-hosted “off-the-shelf” models on their own data and expected behavior for increased performance at a lower cost.

OpenAI likely recognized a huge opportunity for cost savings, given that most use is ChatGPT use, and most of that is basic conversation or requests.

The SLM Competition Heats Up

According to benchmarks, Llama 3.1 70B performs almost as well as GPT-4o on evaluations, and an even larger 405B version actually performs better than GPT-4o in most evaluations, according to leaked evaluations. This means that consumers can increasingly unlock state-of-the-art LLMs in the open realm, meaning they can modify, fine-tune, and adjust them, self-host, and deploy however they want.

With chip architectures rapidly improving, inference providers increasing their speeds with software advancements, and new companies providing effective, convenient, and cost-effective services, the open-source SLM space is advancing faster than ever before. Meanwhile, Apple is copying Meta's playbook planning to use performance on-device, small local models to power Apple intelligence, and minimize the dependency on OpenAI's GPT.

This advancement is great for end users and consumers, but we expect the landscape to continue shifting as consumers keep voting with their wallets.

FOR EMPLOYERS
Job Candidates

Request an intro to candidates - jobs@thetech.buzz.

1. Struck Capital Venture Associate, ex-West2East (Russell Wilson) Director of Special Projects, ex-Phoenix Holdings Investment Associate. Harvard BA. Vietnamese pro basketball player.

2. Lockheed Martin Software Engineer, ex-Rutgers Junior Full Stack Developer. Secret clearance. Expertise in C++, PHP, SQL. Rutgers BA in Computer Science.

FOR INVESTORS
Open Deals

Request an intro to founders - invest@thetech.buzz

1. AI Agent Employees - New York - Sophisticated workflow automation for businesses towards fully autonomous organizations. Previously-exited founder. Backed by Tribe & Kenetic. (Seed)

2. Outbound B2B Campaign AI Co-Pilot - Lexington, MA - Turn any CRM into an AI optimized sales agent for personalized customer campaigns. Founder previous exit to PE. Notable Angel on board. Top Hubspot App. Post revenue.(Seed)

3. Therapy for Baldness - Los Angeles, CA - Developed by ex-Kythera R&D Leader. Virtual development model and world class team seeking to fund POC study. (Seed)

TRENDING TOOLS AND BUZZY TECH
Tools

Google Cloud - Market maker Hudson River joins to access AI chips.

Mistral x Nvidia - Nemo 12B multilingual model supports 128k contexts.

Meta x EssiLux - 1. Meta to buy stake | 2. Google eyes Gemini AI glasses partnership. | 3. New-gen glasses outsell V1 in 6 mths.

Tech

Electrostatic Drone - 4.21g light drone can fly as long as the sun shines.

Meltdown-Proof Nuclear - Chinese reactor completely meltdown-proof.

MIT - AI can predict and reduce semiconductor and insulator waste heat.  

Robotics - 1. Coatue releases “Path to General Purpose Robotics” Report | 2. Tesla to use humanoid robots internally in factories from next year.

DEALS WE’RE WATCHING
Closed Deals:

Enterprise AI - Toronto AI startup Cohere raised $500M growth funding from Nvidia, Salesforce, Cisco, AMD, and Fujitsu at $5.5b, over 2x its 2023 valuation.

Next-gen Cardiovascular Drugs - Boston biotech startup Cardurion Pharmaceuticals secured a $260M Series B led by Ascenta Capital.

Autonomous Navy Vessels - Austin startup Saronic raised a $175M Series B funding round led by Andreessen Horowitz, at a $1b valuation.

Deals to Watch:

AI-Powered Code Editor - A16z- and openAI-backed New York AI startup Anysphere is raising a new round with a valuation of at least $400M.

Nirvana Labs - The LA Web3-native cloud service provider’s revenue is up 650% this year, as it raises a $4M seed following a $1.2M seed in March.

Reddit Sports - Reddit has partnered with the NFL, NBA, and MLB to obtain videos and content, aiming to attract advertisers and share the revenue with the leagues.

Funding Buzz Watch:

NEA - Raising a $540M continuation fund with Goldman Sachs’s Alternatives leading a consortium of LPs backing the fund which will hold NEA’s portfolio stakes in 11 companies, incl. Databricks, Plaid, and Tempus.