Products Overview
Our goal is to create a comprehensive ecosystem that simplifies and enhances AI development, from local devices to server and cloud solutions. We're working on a software suite that would allow the integration of AI into nearly all software, empowering developers to create smarter, more dynamic applications.
Some concrete features of our software include:
- Privacy-first configurations. Allow configurable deployment models to ensure that sensitive data remains secure:
- Edge inference. Perform all AI operations locally on the user's device, maximizing privacy and reducing dependency on external resources.
- Private server inference. Support server deployment and app-specific VPNs configured by the user, ensuring that sensitive data never leaves the user's control.
- Trusted cloud execution. Utilize a globally distributed fleet of cloud servers with trusted execution environments (TEEs), providing hardware-level guarantees against unauthorized access—even in the event of physical tampering.
- Seamless access to a variety of models across platforms. Support a wide range of open-weight models, with a unified API for both low-level features like KV-cache and sampler tweaks, and advanced ones like Retrieval-Augmented Generation (RAG), Tree-of-Thoughts Reasoning, Agentic Workflow Orchestration
- Advanced model selection and routing. Have a sophisticated model router that can automatically select the best model for a given task based on multiple criteria. Continuously update to incorporate the latest models, ensuring that applications can stay at the cutting edge without requiring manual intervention.
The software suite is modular, allowing developers to pick and choose the components that best suit their needs.
Inference Libraries
At the heart of the ecosystem are the Inference Libraries, which provide the core functionality for AI inference. These libraries are designed to be modular and extensible, allowing developers to easily integrate them into their applications. They support a wide range of models and algorithms, enabling developers to build AI applications that are both powerful and flexible.
Inference Interfaces
The Inference Interfaces provide a unified API for interacting with the Inference Libraries. They abstract away the complexities of the underlying libraries, making it easy for developers to integrate AI functionality into their applications. The interfaces support a variety of input and output formats, enabling developers to work with different types of data.
The Interfaces may also be used to abstracta way existing services like OpenAI's GPT or others.
Plugins
Plugins bundle inference libraries and interfaces together, providing a convenient way for developers to access and use them directly in Edge Apps.
Acord Daemon
The Acord Daemon is a desktop-based service with a WebSocket interface. It runs inference libraries locally and acts as a bridge between computational and storage resources of the host machine and Local Apps. Local Apps leverage the daemon for their AI needs.
Server
The Server is designed run as a cloud service, optionally within a trusted execution environment for enhanced security. It offers REST, WebSocket, and custom interfaces, enabling developers to integrate server-based inference into their applications. Unlike Acord, it has features for load balancing, horizontal, and vertical scaling. It powers Client Apps which communicate with a server for their AI needs.
Router
The Router is an intelligent decision-making layer within the ecosystem. Leveraging information about the client and a Model Repository, it determines the optimal approach for handling inference requests. For example, it can decide whether to route a request to a local model, a server, or a third-party service, in order to satisfy the request's requirements for quality, speed, and privacy.
API
The API is a high-level interface that abstracts away the complexities of the underlying system. It provides a seamless way for applications to interact with the ecosystem, regardless of whether they are running on the edge, locally, or in the cloud. The API is designed to be easy to use and flexible, enabling developers to quickly integrate AI functionality into their applications.