cli_buddy 0.1.1

A CLI tool to call any AI to ask, auto-complete, and generate code directly from any command line tool.

---

cli_buddy

---

Call Any LLM from Your Terminal

• Lightweight and Fast
• Supports over 100+ LLMs (GPT, Claude, Cohere, LLama 3, Gemini, Mistral, DeekSeek, and many other LLMs including all Open Source LLMs)
• Use your own API key (OpenRouter)
• Complete privacy - not a single analytic data is collected
• Works on Linux, macOS, and Windows
• Can also view GUI similar to many AI chat app if you want

Demo

Features

1. Code Generation: Ask for code and directly generate (or overwrite) a file with it.
   • Also available: option to copy to clipboard, ask for explanation, or chat with the context.
2. Shell Command Execution: Ask for a shell command and directly execute it in your terminal.
   • Also available: option to copy to clipboard or ask for explanation.
3. Continuous Chat: Chat with AI continuously in your terminal.
4. Session Management: Save chat history in JSON format locally and load it when starting a new session using -s <session_id>.
   • By default, saving chat history is disabled.
5. OpenRouter APIs: Even though it's made as a CLI tool, this package can also be used for any Dart/Flutter project to use OpenRouter APIs. This package exposes OpenRouter APIs and Models. Simply declare final openRouter = OpenRouterService(); and use it in your project. such as final result = await openRouter.invoke(session: ChatSession());

Getting Started 🚀

Installation

1. Installing as a package

   • Simply run:

   dart pub global activate cli_buddy
   

2. Installing Dart SDK

   • Install Dart SDK following the official Install Dart SDK document.
   • Check if Dart sdk is installed by running dart --version
   • Run dart pub global activate cli_buddy

3. Using binary

   • Download binary for your OS in /release (currently I only have MacOS so tested on MacOS but technically should work on all OS)
   • If your system is Unix (MacOS, Linux) run sudo chmod +x <path/to/binary>
   • Test if it runs: ./<path/to/binary> --help
   • Set alias according to your system (e.g. alias buddy='./<path/to/binary>')

Get API Key from OpenRouter

OpenRouter provides unified LLM APIs at (almost) the same cost as the original LLM API provider. Note: OpenRouter provides some free AI models with limitations. You can try out for free using those models without payments Note: I am not affliated with OpenRouter and I do not receive any benefits from them by any means.

Add API Key or create secret.env

Easy way

buddy set -k <your openrouter key>

This will create secret.env and buddy.config files at the default path.

Or

Create a secret.env file in the any directory and add:

openrouter_key=YOUR_OPENROUTER_KEY

Then run:

buddy set -s <path/to/secret.env>

Example buddy.config

{
   "secret_env_path":"secret.env",
   "save_session":false,
   "max_messages":20,
   "default_model":"openai/gpt-4",
   "temperature":0.3,
   "max_tokens":null,
   "top_p":null,
   "top_k":null,
   "frequency_penalty":null,
   "presence_penalty":null,
   "repetition_penalty":null,
   "min_p":null,
   "top_a":null,
   "seed":null,
   "logit_bias":null,
   "logprobs":null,
   "top_logprobs":null,
   "response_format":null,
   "stop":null,
   "cmd_prompt":"If there is a lack of details, provide most logical solution.\nEnsure the output is a valid shell command.\nIf multiple steps required try to combine them together in one command.\nProvide only plain text without Markdown formatting.\nDo not provide markdown formatting such as ```.\n",
   "explain_prompt":"Provide short and concise explanation of your previous response about command or code.\nProvide only plain text without Markdown formatting.\nDo not provide markdown formatting such as ```\n",
   "code_prompt":"Provide only code as output without any description.\nProvide only code in plain text format without Markdown formatting.\nDo not include symbols such as ``` or ```python.\nIf there is a lack of details, provide most logical solution.\nYou are not allowed to ask for more details.\nFor example if the prompt is \"Hello world Python\", you should return \"print('Hello world')\".\n",
   "chat_prompt":"Provide concise response unless asked for more details.\nAvoid using any markdown formatting such as ```, *, #.\n"
}

Basic Usage

# Shell command
$ buddy shell how can I update homebrew

# If you want to use special characters, use quotes
$ buddy shell "how can I update homebrew?"

# Coding
$ buddy code "Generate a random number between 1 and 10 in Dart"

# Chat
$ buddy chat "Tell me about the SSH protocol"

# Show Graphical User Interface
$ buddy open

# Show CLI version
$ buddy --version

# Show usage help
$ buddy --help

Info Command

The info command allows you to search for AI models in OpenRouter. Below are the available options and flags:

• -q, --query: Search for AI models by name, provider, or model type (e.g., text, image).
• -o, --order: Specify the order in which the results should be displayed. Allowed values: name, context, prompt, completion, image.
• -f, --config: Display the current configuration file. If it does not exist, create a new one.
• -c, --credits: Display the credits available in OpenRouter.
• -l, --list: List all AI models available in OpenRouter.
• -p, --parameters: Query the parameters of a specific AI model.
• -s, --sessions: (-s list) List saved chat histories in the default session folder or (-s <session_id>) view a specific chat history.

Example Usage

# Search for AI models by name
$ buddy info -q "claude"

# Search for AI models by provider
$ buddy info -q "openai"

# List all available AI models
$ buddy info -l

# Display the current configuration
$ buddy info -f

# Display the credits used for your OpenRouter account
$ buddy info -c

# Query the parameters of a specific AI model
$ buddy info -p "model_id"

# List saved chat histories
$ buddy info -s list

# View a specific chat history by session ID
$ buddy info -s "session_id"

Set Command

The set command allows you to set or update configuration values. Below are the available options and flags:

• -s, --secret-path: Specify the path to the secret.env file in the buddy.config.
• -k, --api-key: Set the API key in the secret.env file. If the file doesn't exist, it will be created.
• -r, --remove-sessions: Remove all saved sessions.
• -m, --model: Set the default AI model to be used.
• -e, --save-session: Enable or disable session saving. Allowed values: true, false.
• -a, --max-messages: Set the maximum number of messages to retain.
• -t, --temperature: Set the temperature for AI responses (controls randomness).
• -x, --max-tokens: Set the maximum number of tokens per response.
• --top-p: Set the top-p sampling parameter.
• --top-k: Set the top-k sampling parameter.
• --freq-penalty: Set the frequency penalty for repetitive responses.
• --presence-penalty: Set the presence penalty for repetitive responses.
• --repetition-penalty: Set the penalty for repeating the same phrases.
• --min-p: Set the minimum probability parameter.
• --top-a: Set the top-a sampling parameter.
• --seed: Set the seed for random number generation.
• --logit-bias: Set the logit bias for specific tokens.
• --logprobs: Set the number of log probabilities to return.
• --top-logprobs: Set the number of top log probabilities to return.
• --response-format: Set the format for AI responses.
• --stop-seq: Set the sequences where the AI should stop generating responses.
• --cmd-prompt: Set the command prompt template which is used as a system message.
• --explain-prompt: Set the explanation prompt template which is used as a system message.
• --code-prompt: Set the code generation prompt template which is used as a system message.
• --chat-prompt: Set the chat prompt template which is used as a system message.

Example Usage

# Set the API key
$ buddy set -k "your_openrouter_key"

# Set the path to the secret.env file (if you want to save it in the another directory)
$ buddy set -s "/path/to/secret.env"

# Set the default AI model
$ buddy set -m "openai/gpt-3.5-turbo"

# Enable session saving
$ buddy set -e true

# Set the maximum number of messages to retain as context to provide to the AI
$ buddy set -a 50

# Set the temperature for AI responses
$ buddy set -t 0.7

# Set the maximum number of tokens per response (by default it's set to null and API will use the maximum number of tokens)
$ buddy set -x 4000

# Remove all saved sessions
$ buddy set -r

Markdown format

• As for 'code', 'suggest', 'explain' mode, by default system prompt will discourage using markdown format, will remove it in the final output (for 'code copy/shell run/copy').
• As for 'chat' mode, markdown is allowed, but to use it, you need to pass '-m --markdown' flag. However, if markdown flag is used, real time streaming message will be disabled. Also implementing markdown in console is hard. Feel free to make a merge request if you did it.

Roadmap

• Writing code to a file then executing
• Indexing specified path and file extension for Rag
• Support for RAG (Retrieval-Augmented Generation)
• Automated comments for all speficied file path/file extension
• Automated improving all specified file path/file extension
• Support for Google Vertex AI API

Contributions

Contributions are welcome! Feel free to open issues or submit pull requests.

---

To-Dos

1. If new empty session is alive and page is refreshed, prevent another new session being creating in the sidebar
2. add method to remove single session in the side bar
3. add configuration button and popup to change config
4. add a button which will populate the chat area to display all available models and when selected it will show a confirmation prompt and set it as default model
5. add a button that will show a popup to configure parameters
6. allow chaging the system prompt as well as user + assistant prompts per session
7. add a button that will show a popup to set default system prompts