Cerebras
September 14, 2024
Cerebras Unicorn News - September 14, 2024
Cerebras has introduced a new LLM plugin on PyPI, enabling fast inference API support for their hosted models. This plugin supports models like llama3.1-8b and llama3.1-70b, offering users extensive customization options. Learn how to set up and utilize this powerful tool.
() Introduction
() Cerebras Enhances AI Model Accessibility with New LLM Plugin on PyPI
Cerebras has introduced a new plugin for Large Language Models (LLM) that enhances fast inference API support for their hosted models. This plugin, available on PyPI, allows users to prompt Cerebras models by installing it in the same environment as LLM. To use the plugin, users must obtain a Cerebras API key and configure it with the command 'llm keys set cerebras'. The plugin supports models such as llama3.1-8b and llama3.1-70b, and provides options to control parameters like temperature, max_tokens, top_p, and seed, offering significant customization. For local setup, users need to check out the code, create a virtual environment, install necessary dependencies, and run tests using pytest to ensure proper functionality.
Sources
