9 Commits

Author SHA1 Message Date
darthrubik c0559f7d6a Unhardcode build directory 2026-05-13 14:08:26 -05:00
darthrubik 6341e3cd8c Adding another test 2026-05-13 11:48:07 -05:00
darthrubik 3943fa1fc3 Removing unneeded printing 2026-05-13 11:44:11 -05:00
darthrubik 2866f970b1 Making the tests actually test 2026-05-13 11:43:13 -05:00
darthrubik a64dd2bfa4 Removing tmp dirs from tests 2026-05-13 11:15:55 -05:00
darthrubik 2abfc88899 Removing weird test design decisions 2026-05-13 11:12:53 -05:00
darthrubik ae171c83c1 Cleanup 2026-05-13 11:05:21 -05:00
darthrubik 1a45faaf20 WIP 2026-05-13 08:25:24 -05:00
darthrubik 6bdc34a3cb Create PRD 2026-05-13 07:35:59 -05:00
11 changed files with 395 additions and 26 deletions
+1
View File
@@ -28,6 +28,7 @@ add_library(drharness_lib
src/ollama_client.cpp
src/harness.cpp
src/debug_client.cpp
src/editor.cpp
${TOOL_SOURCES}
)
target_include_directories(drharness_lib PUBLIC include tools)
+122
View File
@@ -0,0 +1,122 @@
# PRD: Editor-Based Prompt Input for drharness
## Problem Statement
Currently, the `drharness` agentic tool operates on a strict turn-based CLI paradigm where the user must provide every user message via the `-m` argument. This becomes cumbersome for iterative conversations, as it requires repetitive typing in the terminal and offers no visual context of the AI's previous response during the input phase. The user lacks an interactive, reviewable interface for crafting prompts.
## Solution
Introduce an interactive editor mode to `drharness`. If the user omits the `-m` flag, the tool will open an external text editor preloaded with the AI's last response (or full history). The user can review the context and type their reply in the editor. The tool parses the edited file to extract the new prompt, enabling a seamless, Git-commit-message-style workflow for agentic interactions.
## User Stories
1. As a user, I want to run `drharness` without `-m` so that it opens an editor for me to type my prompt, rather than typing directly in the terminal.
2. As a user, I want the editor to be preloaded with the AI's last message so that I have immediate context of what I am replying to.
3. As a user, I want the editor to use a "git scissors" separator (`-- >8 --`) so that I can clearly distinguish between the AI's context and my new input.
4. As a user, I want the system to extract only the text below the separator so that any accidental edits to the AI's context are ignored.
5. As a user, I want to use the `$VISUAL` environment variable to choose my preferred editor so that the tool integrates with my existing workflow.
6. As a user, I want the tool to exit immediately if I cancel the editor or leave the prompt empty so that I don't accidentally send an incomplete or invalid message to the AI.
7. As a user, I want to see a helpful placeholder message when starting a new conversation so that I know how to use the editor mode correctly.
8. As a user, I want to use the `-f` flag to preload the entire chat history in the editor so that I can review the full context of a long conversation before replying.
9. As a user, I want the full history to be displayed in plain text without terminal color codes so that it is readable and editable in any text editor.
10. As a user, I want the tool to work with the debug client (`-d`) so that I can test editor interactions without needing a live AI model.
11. As a user, I want the JSON state file to be updated only after a successful AI response so that my editor edits are not lost or corrupted by failed API calls.
12. As a developer, I want the editor logic encapsulated in a single deep module so that the workflow is testable and maintainable.
13. As a developer, I want the prompt parsing logic to be isolated so that the "scissors" parsing behavior can be tested independently of the file I/O.
14. As a user, I want the tool to fallback to `$EDITOR` if `$VISUAL` is not set so that it works with standard Unix editor configurations.
15. As a user, I want the tool to exit with an error if neither `$VISUAL` nor `$EDITOR` is set so that I am not stuck in a broken state.
16. As a user, I want the `-m` flag to still take precedence over the editor so that I can bypass the editor when needed for scripting.
17. As a user, I want the last message from the AI to be included in the editor even if it contains reasoning content so that I see the full thought process.
18. As a user, I want the tool to handle the "first turn" gracefully by providing specific instructions on how to use the editor.
19. As a user, I want the history to be formatted consistently with the `-p` print command so that my reading experience is familiar.
20. As a user, I want the parser to find the *last* instance of the separator so that if the AI's output contains the separator string, it is handled correctly.
## Implementation Decisions
### Modules
1. **Editor Module (`editor.hpp/cpp`)**
* **Role:** Deep module encapsulating the entire editor workflow.
* **Responsibilities:**
* Reads `HarnessState` from JSON.
* Formats history (Last Message or Full History) using the refactored print logic.
* Launches the external editor via system call.
* Reads the modified file back.
* Parses the file to extract the user prompt using the `-- >8 --` separator.
* Validates the prompt (non-empty).
* **Interface:** `std::string run_editor(HarnessState state, const std::string& file_path, bool full_history)`
* **Dependencies:** Ollama Client (for print logic), File System.
2. **Ollama Client Module (`ollama_client.hpp/cpp`)**
* **Role:** Refactors the history printing utility.
* **Responsibilities:**
* Updates `print_history` to accept `std::ostream&` and `bool colored`.
* Provides a plain-text version for the editor preload.
* **Interface:** `void print_history(std::ostream& out, const nlohmann::json& data, bool colored = true)`
* **Dependencies:** None.
3. **Main CLI Module (`main.cpp`)**
* **Role:** Orchestrates the application flow.
* **Responsibilities:**
* Parses CLI arguments (adds `-f` flag).
* Checks for `-m`. If missing, calls `Editor Module`.
* Validates editor result.
* Proceeds with `Harness` chat.
* **Interface:** Modified `main()` function.
* **Dependencies:** Editor Module, Harness, Tool Registry.
### Technical Clarifications
* **Separator:** The delimiter is a single line containing `-- >8 --` (trimmed). The parser finds the *last* occurrence to ensure robustness.
* **Editor Selection:** Strict adherence to `$VISUAL` -> `$EDITOR`. No fallback to `vi`. Exit on missing editor.
* **Validation:** Empty or whitespace-only prompts result in an immediate exit (abort).
* **State Management:** JSON file is only written to *after* a successful `harness.chat()` call.
### API Contracts
* **Editor Module Interface:**
```cpp
std::string run_editor(HarnessState state, const std::string& file_path, bool full_history);
// Returns the extracted user prompt.
// Throws std::runtime_error if editor fails or prompt is empty.
```
## Testing Decisions
### Testing Strategy
Tests will focus on the **Editor Module** as it contains the core new logic. We will use Bash-based tests (consistent with the existing `tests/` directory) to simulate the editor behavior.
### Test Cases
1. **Mock Editor Success:**
* Set `$EDITOR` to a script that writes a known prompt to the file.
* Run `drharness` without `-m`.
* Verify the harness proceeds with the chat using the known prompt.
2. **Mock Editor Empty Input:**
* Set `$EDITOR` to a script that writes only the separator (no user text).
* Run `drharness`.
* Verify the harness exits with an error.
3. **Full History Mode:**
* Run `drharness -f`.
* Verify the file content preloaded includes the full history formatted as text.
4. **Last Message Mode:**
* Run `drharness` (no `-f`).
* Verify the file content preloaded includes only the last AI message.
5. **Separator Parsing:**
* Verify that text above `-- >8 --` is discarded.
* Verify that text below `-- >8 --` is extracted, even if multiple separators exist.
6. **First Turn:**
* Verify the placeholder text is shown for a new conversation.
### Prior Art
* Existing Bash tests in `tests/` (e.g., `test_n_flag.sh`, `test_smoke_basic.sh`) serve as the model for structure and execution.
* The `run_test.sh` script will be used to orchestrate these new tests.
## Out of Scope
* Support for Windows (where `$VISUAL`/`$EDITOR` behavior differs and shell tools are restricted).
* Rich text formatting in the editor (plain text only).
* Real-time preview of the AI response while editing.
* Support for editors that do not exit with a standard success code (non-POSIX editors).
## Further Notes
* The "git scissors" concept (`-- >8 --`) is chosen for its familiarity to developers and its effectiveness as a parsing boundary.
* The refactoring of `print_history` allows reuse of the formatting logic for both terminal display (`-p`) and editor preload, ensuring consistency.
* The Editor Module is designed as a "deep module" to encapsulate complexity, making it highly testable and isolating it from the rest of the CLI logic.
+5
View File
@@ -0,0 +1,5 @@
#pragma once
#include "json_types.hpp"
#include <string>
std::string run_editor(HarnessState state, const std::string& file_path, bool full_history);
+2 -1
View File
@@ -4,6 +4,7 @@
#include <stdexcept>
#include <string>
#include <vector>
#include <iostream>
struct OllamaConfig {
std::string base_url = "https://192.168.8.63/ollama";
@@ -37,4 +38,4 @@ private:
};
void print_history(nlohmann::json const& data);
void print_history(std::ostream& out, const nlohmann::json& data, bool colored = true);
+91
View File
@@ -0,0 +1,91 @@
#include "editor.hpp"
#include "ollama_client.hpp"
#include <fstream>
#include <iostream>
#include <sstream>
#include <cstdlib>
#include <string>
#include <algorithm>
#include <filesystem>
static std::string trim(const std::string& s) {
auto start = s.find_first_not_of(" \t\n\r");
return (start == std::string::npos) ? "" : s.substr(start);
}
std::string run_editor(HarnessState state, const std::string& file_path, bool full_history) {
std::ofstream file(file_path);
if (!file.is_open()) {
throw std::runtime_error("Failed to open temp file for editor.");
}
if (state.history.empty()) {
file << "New Conversation.\n\n";
file << "This is the start of a new chat. Type your prompt below the separator.\n";
file << "The AI will respond after you save and close the editor.\n\n";
} else {
nlohmann::json j = nlohmann::json(state);
if (full_history) {
print_history(file, j, false);
} else {
// Find the last AI response
const Message* last_ai_msg = nullptr;
for (int i = static_cast<int>(state.history.size()) - 1; i >= 0; --i) {
if (state.history[i].role == "assistant") {
last_ai_msg = &state.history[i];
break;
}
}
if (last_ai_msg) {
file << "[" << last_ai_msg->role << "] ";
if (!last_ai_msg->reasoning.empty()) {
file << "[Thinking: " << last_ai_msg->reasoning << "] ";
}
file << last_ai_msg->content << "\n\n";
} else {
// Fallback to last message if no assistant message found
const auto& last_msg = state.history.back();
file << "[" << last_msg.role << "] " << last_msg.content << "\n\n";
}
}
}
file << "-- >8 --\n\n";
file.close();
const char* editor_env = getenv("VISUAL");
if (!editor_env) editor_env = getenv("EDITOR");
if (!editor_env) {
throw std::runtime_error("Neither $VISUAL nor $EDITOR is set.");
}
std::string cmd = std::string(editor_env) + " " + file_path;
int res = system(cmd.c_str());
if (res != 0) {
throw std::runtime_error("Editor exited with error.");
}
std::ifstream infile(file_path);
std::string content((std::istreambuf_iterator<char>(infile)), std::istreambuf_iterator<char>());
size_t pos = content.find("-- >8 --");
if (pos == std::string::npos) {
throw std::runtime_error("Separator `-- >8 --` not found.");
}
size_t last_pos = std::string::npos;
while (pos != std::string::npos) {
last_pos = pos;
pos = content.find("-- >8 --", pos + 1);
}
std::string extracted = content.substr(last_pos + std::string("-- >8 --").size());
extracted = trim(extracted);
if (extracted.empty()) {
throw std::runtime_error("Prompt is empty after separator.");
}
return extracted;
}
+25 -8
View File
@@ -1,6 +1,7 @@
#include "harness.hpp"
#include "ollama_client.hpp"
#include "debug_client.hpp"
#include "editor.hpp"
#include "tool_registry.hpp"
#include "agent_profile.hpp"
#if !defined(_WIN32)
@@ -17,6 +18,7 @@
#include <fstream>
#include <string>
#include <vector>
#include <filesystem>
static const std::vector<AgentProfile> profiles = {
#if !defined(_WIN32)
@@ -77,6 +79,7 @@ int main(int argc, char** argv) {
bool list_models = false;
char const* print_file = nullptr;
std::vector<std::string> tools;
bool full_history = false;
consume_arg(); // progname
@@ -130,6 +133,11 @@ int main(int argc, char** argv) {
}
exit(0);
}
else if (argv[0] == std::string("-f"))
{
consume_arg();
full_history = true;
}
else if (argv[0] == std::string("-h"))
{
std::cout << "Usage: drharness [options]\n"
@@ -140,6 +148,7 @@ int main(int argc, char** argv) {
<< " -n Start a new conversation\n"
<< " -m <prompt> Prompt message (required)\n"
<< " -M <model> Ollama model name\n"
<< " -f Open editor with full history (requires -F)\n"
<< " -l List available tools\n"
<< " -L List available models on server side\n"
<< " -D <debug-json> (Alias for -d in future) / -d Use debug client (mock API via stdin)\n"
@@ -181,7 +190,7 @@ int main(int argc, char** argv) {
try {
nlohmann::json data = nlohmann::json::parse(file);
print_history(data);
print_history(std::cout, data);
} catch (const nlohmann::json::parse_error& e) {
std::cerr << "Error parsing JSON: " << e.what() << "\n";
return 1;
@@ -200,12 +209,6 @@ int main(int argc, char** argv) {
exit(1);
}
if (!prompt)
{
std::cerr << "Prompt is required (-m)" << std::endl;
exit(1);
}
if (start_new) {
std::ifstream file(file_to_open, std::ios::ate);
if (file.is_open()) {
@@ -232,7 +235,21 @@ int main(int argc, char** argv) {
std::cerr << "drharness | model: " << cfg.model
<< " | " << cfg.base_url << "\n\n";
auto reply = harness.chat(prompt);
std::string final_prompt;
if (prompt) {
final_prompt = prompt;
} else {
std::string tmp_path = std::filesystem::temp_directory_path() / "drharness_prompt.txt";
try {
final_prompt = run_editor(harness.serialize(), tmp_path, full_history);
} catch (const std::runtime_error& e) {
std::cerr << "Editor error: " << e.what() << "\n";
exit(1);
}
}
auto reply = harness.chat(final_prompt);
std::ofstream harness_state(file_to_open);
std::cout << reply << std::endl;
+28 -15
View File
@@ -118,7 +118,7 @@ std::vector<std::string> OllamaClient::list_models() const {
}
void print_history(nlohmann::json const& data)
void print_history(std::ostream& out, const nlohmann::json& data, bool colored)
{
auto state = data.get<HarnessState>();
auto history = state.history;
@@ -127,39 +127,52 @@ void print_history(nlohmann::json const& data)
std::string role = msg.role;
if (role == "user") {
std::cout << "\033[36m[User]\033[0m " << msg.content << "\n\n";
if (colored) out << "\033[36m[User]\033[0m ";
else out << "[User] ";
out << msg.content << "\n\n";
}
else if (role == "assistant") {
// Check for explicit reasoning/thinking field
if (!msg.reasoning.empty()) {
std::cout << "\033[32m[Ai Thinking]\033[0m " << msg.reasoning << "\n\n";
if (colored) out << "\033[32m[Ai Thinking]\033[0m ";
else out << "[Ai Thinking] ";
out << msg.reasoning << "\n\n";
}
if (!msg.reasoning_content.empty()) {
std::cout << "\033[32m[Ai Thinking]\033[0m " << msg.reasoning_content << "\n\n";
if (colored) out << "\033[32m[Ai Thinking]\033[0m ";
else out << "[Ai Thinking] ";
out << msg.reasoning_content << "\n\n";
}
// Check for tool calls
if (!msg.tool_calls.empty()) {
std::cout << "\033[33m[Ai Tool Call]\033[0m\n";
if (colored) out << "\033[33m[Ai Tool Call]\033[0m\n";
else out << "[Ai Tool Call]\n";
for (const auto& tc : msg.tool_calls) {
std::cout << " - " << tc.function.name << "(" << tc.function.arguments << ")\n";
out << " - " << tc.function.name << "(" << tc.function.arguments << ")\n";
}
}
if (!msg.content.empty())
std::cout << "\033[37m[Ai Response]\033[0m " << msg.content << "\n\n";
if (!msg.content.empty()) {
if (colored) out << "\033[37m[Ai Response]\033[0m ";
else out << "[Ai Response] ";
out << msg.content << "\n\n";
}
}
// Handle cases where thinking might be a separate message type (reasoning/thinking roles)
else if (role == "reasoning" || role == "thinking") {
std::cout << "\033[32m[Ai Thinking]\033[0m " << msg.content << "\n\n";
if (colored) out << "\033[32m[Ai Thinking]\033[0m ";
else out << "[Ai Thinking] ";
out << msg.content << "\n\n";
}
else if (role == "tool") {
std::cout << "\033[35m[Tool Result]\033[0m " << msg.content << "\n\n";
if (colored) out << "\033[35m[Tool Result]\033[0m ";
else out << "[Tool Result] ";
out << msg.content << "\n\n";
}
else {
std::cout << "\033[90m[" << role << "]\033[0m " << msg.dump(2) << "\n\n";
if (colored) out << "\033[90m[" << role << "]\033[0m ";
else out << "[" << role << "] ";
out << msg.dump(2) << "\n\n";
}
}
std::cout << "\033[0m"; // Reset colors
if (colored) out << "\033[0m";
}
+4 -1
View File
@@ -5,7 +5,7 @@ function(add_dr_test script)
set_tests_properties(${script} PROPERTIES
TIMEOUT 30
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
endfunction()
@@ -26,3 +26,6 @@ add_dr_test(test_n_flag)
add_dr_test(test_load_chat)
add_dr_test(process_reasoning)
# Add editor tests
add_dr_test(test_editor)
+26
View File
@@ -0,0 +1,26 @@
#!/bin/bash
# $1 is the file path passed to the editor
FILE=$1
# For inspecting by tests
cat $FILE > hist.md
MODE="${DRHARNESS_EDITOR_MODE:-default}"
if [[ "$MODE" == "empty" ]]; then
echo "-- >8 --" > $FILE
elif [[ "$MODE" == "full_history" ]]; then
echo "" >> $FILE
echo "My Full History Reply" >> $FILE
elif [[ "$MODE" == "separator" ]]; then
echo "Context" > $FILE
echo "-- >8 --" >> $FILE
echo "-- >8 --" >> $FILE
echo "Parsed Prompt" >> $FILE
elif [[ "$MODE" == "placeholder" ]]; then
echo "Placeholder text" > $FILE
echo "-- >8 --" >> $FILE
echo "New Chat Reply" >> $FILE
else
echo "" >> $FILE
echo "Default Prompt" >> $FILE
fi
+3 -1
View File
@@ -68,7 +68,8 @@ export -f dr_result
# Resolve paths relative to this script
SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
PROJECT_ROOT=$(cd "${SCRIPT_DIR}/.." && pwd)
DRHARNESS="${PROJECT_ROOT}/build/drharness"
DRHARNESS="${PWD}/drharness"
MOCK_EDITOR="${PROJECT_ROOT}/tests/mock_editor.sh"
# Validate drharness binary exists
if [ ! -f "${DRHARNESS}" ]; then
@@ -89,6 +90,7 @@ TEST_TEMP_DIR=$(mktemp -d /tmp/drharness_test_XXXXXX)
# Export variables for the test script
export DRHARNESS
export TEST_TEMP_DIR
export MOCK_EDITOR
# Cleanup function
cleanup() {
+88
View File
@@ -0,0 +1,88 @@
#!/bin/bash
set -e
export EDITOR="$MOCK_EDITOR"
# Helper to create a harness state file
create_state() {
local file=$1
local history_json=$2
echo "{" > "$file"
echo " \"history\": $history_json," >> "$file"
echo " \"tools\": []," >> "$file"
echo " \"config\": {" >> "$file"
echo " \"max_tool_iterations\": 16," >> "$file"
echo " \"verbose\": false," >> "$file"
echo " \"depth\": 0" >> "$file"
echo " }" >> "$file"
echo "}" >> "$file"
}
create_state test_drharness.json '[{"role": "assistant", "content": "This is the AI last response."}]'
export DRHARNESS_EDITOR_MODE="default"
be_done 'Mock response' | $DRHARNESS -F test_drharness.json -d
# Make sure we got correct prompt out of it
if [[ "$(dr_result test_drharness.json 1)" != "Default Prompt" ]]; then
exit 1
fi
rm -f test_drharness_empty.json
create_state test_drharness_empty.json '[{"role": "assistant", "content": "Mock response"}]'
export DRHARNESS_EDITOR_MODE="empty"
if $DRHARNESS -F test_drharness_empty.json -d < /dev/null 2>&1 | grep -q "Prompt is empty"; then
echo "PASS: Empty input handled"
else
echo "FAIL: Empty input not handled correctly"
exit 1
fi
rm -f test_drharness_sep.json
create_state test_drharness_sep.json '[{"role": "assistant", "content": "Mock response"}]'
export DRHARNESS_EDITOR_MODE="separator"
be_done 'Mock response' | $DRHARNESS -F test_drharness_sep.json -d
# Make sure we got correct prompt out of it
if [[ "$(dr_result test_drharness_sep.json 1)" != "Parsed Prompt" ]]; then
exit 1
fi
cat > expected_history <<EOF
[assistant] Mock response
-- >8 --
EOF
# Check expected history
diff -u expected_history hist.md
rm -f test_drharness_full.json
create_state test_drharness_full.json '[{"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Hi there!"}]'
export DRHARNESS_EDITOR_MODE="full_history"
be_done 'Mock response' | $DRHARNESS -F test_drharness_full.json -f -d
cat > expected_history <<EOF
[User] Hello
[Ai Response] Hi there!
-- >8 --
EOF
# Check expected history
diff -u expected_history hist.md
export DRHARNESS_EDITOR_MODE="default"
be_done 'Mock response' | $DRHARNESS -F test_drharness_first.json -d -n
cat > expected_history <<EOF
New Conversation.
This is the start of a new chat. Type your prompt below the separator.
The AI will respond after you save and close the editor.
-- >8 --
EOF
diff -u expected_history hist.md