Products Overview

Our goal is to create a comprehensive ecosystem that simplifies and enhances AI development, from local devices to server and cloud solutions. We're working on a software suite that would allow the integration of AI into nearly all software, empowering developers to create smarter, more dynamic applications.

Some concrete features of our software include:

• Privacy-first configurations. Allow configurable deployment models to ensure that sensitive data remains secure:
  • Edge inference. Perform all AI operations locally on the user's device, maximizing privacy and reducing dependency on external resources.
  • Private server inference. Support server deployment and app-specific VPNs configured by the user, ensuring that sensitive data never leaves the user's control.
  • Trusted cloud execution. Utilize a globally distributed fleet of cloud servers with trusted execution environments (TEEs), providing hardware-level guarantees against unauthorized access—even in the event of physical tampering.
• Seamless access to a variety of models across platforms. Support a wide range of open-weight models, with a unified API for both low-level features like KV-cache and sampler tweaks, and advanced ones like Retrieval-Augmented Generation (RAG), Tree-of-Thoughts Reasoning, Agentic Workflow Orchestration
• Advanced model selection and routing. Have a sophisticated model router that can automatically select the best model for a given task based on multiple criteria. Continuously update to incorporate the latest models, ensuring that applications can stay at the cutting edge without requiring manual intervention.

The software suite is modular, allowing developers to pick and choose the components that best suit their needs.

Inference Libraries

At the heart of the ecosystem are the Inference Libraries, which provide the core functionality for AI inference. These libraries are designed to be modular and extensible, allowing developers to easily integrate them into their applications. They support a wide range of models and algorithms, enabling developers to build AI applications that are both powerful and flexible.

Standardized Interfaces

The Standardized Interfaces provide a unified API for interacting with the Inference Libraries. They abstract away the complexities of the underlying libraries, making it easy for developers to integrate AI functionality into their applications. The interfaces support a variety of input and output formats, enabling developers to work with different types of data.

The Interfaces may also be used to abstract away existing services like OpenAI's GPT or others.

Plugins

Plugins bundle inference libraries and interfaces together, providing a convenient way for developers to access and use them directly in Edge Apps.

Acord Daemon

The Acord Daemon is a desktop-based service with a WebSocket interface. It runs inference libraries locally and acts as a bridge between computational and storage resources of the host machine and Local Apps. Local Apps leverage the daemon for their AI needs.

Router

The Router is an intelligent decision-making layer within the ecosystem. Leveraging information about the client and a Model Repository, it determines the optimal approach for handling inference requests. For example, it can decide whether to route a request to a local model, a server, or a third-party service, in order to satisfy the request's requirements for quality, speed, and privacy.

In order to satisfy those needs the Router relies on variety of inputs maintained by us, such as benchmark results and a model leaderborad. Access to over the air router model updates is provided as a commercial service by the AlpacaCore team, generating revenues for our community of inference library authors and model creators.

Benchmarking SDK

The standardized interfaces make it possible to execute a wide range of benchmarks against a wide range of inference libraries and models. Users can take advantage of our developed automation to identify which models achieve best results on their private data sets. Model vendors can provide stronger guarantees for the authenticity of their benchmark results by allowing their evaluation to be carried out in a trusted execution environment, powered by the AlpacaCore SDK.

Model Leaderboard

The AlpacaCore team will maintain an open model leaderboard, ensuring that anyone can introduce new models and evaluation sets while delivering strong technical guarantees for the authenticity of the results. Achieving SOTA results in a niche category will ensure that your model starts receiving traffic from the AlpacaCore router, which creates a viable business model for training fine-tuned expert models.