feature/rebase #44

Merged
darthrubik merged 52 commits from feature/rebase into master 2026-07-23 17:03:56 -05:00
Owner
No description provided.
darthrubik added 52 commits 2026-07-23 17:02:49 -05:00
Add rebase parser that validates rebase editor format with:
- RebaseCommand enum (Pick, Edit, Drop)
- RebaseLine struct (command, index, role)
- parse_rebase_file() with strict validation and error aggregation

Validations:
- Invalid commands (not pick/edit/drop)
- Index out of range
- Duplicate indices
- Missing/malformed role brackets
- Invalid role values (must be user/assistant/tool)
- Comment lines (#) and empty lines are skipped

Tests cover all validation scenarios via -P debug flag.
- Added HistoryEditor class with build_rebase_content() method
- Formats messages for rebase editor display:
  - Normal messages show first line of content
  - Assistant messages with tool_calls show [tool_call: function_name]
  - Tool messages show first line of result content
  - Empty messages show (empty)
- Added header comment block explaining pick/edit/drop commands
- Added -B CLI flag for testing
- Added unit tests covering all display format scenarios
Add the -b/--edit-history CLI flag to main.cpp with the following behavior:
- Flag is recognized in both short (-b) and long (--edit-history) forms
- Requires -F <chat_file> to be specified, otherwise prints error and exits
- Rejects empty chat files with error message
- Rejects chat files with no history (empty history array) with error message
- Updates help message to document the new flag

This is a thin slice that validates the CLI integration point before the
full history editing feature is built.

Includes integration tests covering all error cases.
- Added apply_rebase() method to RebaseEngine class
- Pick commands preserve messages unchanged
- Drop commands remove messages from result
- Edit commands update role while preserving all other fields
- Messages not referenced by any command are preserved (implicit pick)
- Lenient mode: orphaned tool results are allowed
- Added -A test flag for testing apply_rebase
- Added unit tests in test_rebase_apply.sh
- Replace .at("field").get<T>() with json_to_field<T>() calls
- Use .contains() and .value() for safe field access
- Follows CONTRIBUTING.md rule to always use json_to<T>() for JSON deserialization
- Add HistoryEditor::build_edit_prompt() to format edit prompts
  with previous message context and >8 separator
- Message 0 shows '(no previous message)' instead of context
- Add -T debug flag to test build_edit_prompt
- Extend mock_editor.sh to support sequential invocations via
  counter file and staged input files
- Add unit tests for build_edit_prompt functionality
- Register new test in CMakeLists.txt
- Add edit_history_interactive() method to HistoryEditor class
  - Implements the complete history editing flow:
    1. Build rebase content from history
    2. Spawn editor for rebase commands
    3. Parse and validate rebase commands
    4. For each edit command, build edit prompt and spawn editor
    5. Apply all changes
  - Returns expected<vector<Message>, string>

- Add content field to RebaseLine struct for storing edited content
- Update apply_rebase() to use content field when present
- Wire up -b/--edit-history flag to call edit_history_interactive()
- Program exits after successful edit and saves modified history

- Add 12 integration tests for the edit history feature:
  1. Pick all messages (history unchanged)
  2. Drop a single message
  3. Edit message content
  4. Change message role
  5. Delete line equals drop (implicit pick)
  6. Edit first message (no context)
  7. Empty history rejected
  8. Invalid command error
  9. Invalid index error
  10. Invalid role error
  11. Multiple edits in one session
  12. Edit then continue conversation normally

- Register test_edit_history in CMakeLists.txt
- Remove namespace alias in src/rebase_engine.cpp (use full std::filesystem:: path)
- Add dr_role() helper function to tests/run_test.sh
- Replace inline JSON parsing in tests/test_edit_history.sh with dr_role() helper
- Updated project description to mention C++23
- Updated build requirements to mention C++23
Create new utility module (include/utility.hpp, src/utility.cpp) containing
common code that was duplicated between rebase_engine.cpp and editor.cpp:

- trim() function
- random_string() function
- random_file_name() function
- Defer RAII helper class
- spawn_editor() function

Both rebase_engine.cpp and editor.cpp now include utility.hpp instead of
duplicating these functions.

All 33 unit tests pass.
The rebase editor now includes a scissors line (-- >8 --) that separates
the comment header from the actual pick commands. This helps editors like
Vim automatically position the cursor at the relevant content and signals
to users where the editable commands begin.

Issue: #104
- Added new test test_no_edits_history_unchanged that verifies when the
  user does nothing with the rebase editor, the history remains unchanged
- Fixed staged input format in all tests to match actual build_rebase_content
  output (pick commands only after -- >8 -- scissors line, not before)
The edit prompt now shows the explanatory text 'Edit message X (role: Y) below:'
BEFORE the scissors (-- >8 --) separator, not after. This ensures the
explanatory text is not part of the editable content.

Also added test_edit_prompt_noop to verify that doing nothing at the edit
prompt stage keeps history unchanged.
- Add dr_reasoning() helper to run_test.sh
- Add dr_reasoning_content() helper to run_test.sh
- Update process_reasoning.sh to use helper functions instead of direct jq calls

This follows CONTRIBUTING.md guidelines that prohibit direct JSON
parsing in test files.
Add create_chat_file() helper to run_test.sh and update test_edit_history.sh
and test_edit_history_flag.sh to use it instead of manually creating JSON.

This follows the CONTRIBUTING.md guideline that tests should use helper
functions instead of inline JSON handling.
Clarify that test flags should be minimal hooks into internal
functionality, not contain test logic. All test logic (assertions,
validation) belongs in Bash test scripts in tests/.

This addresses the ambiguity mentioned in issue 109 regarding
the distinction between test hooks in main.cpp and test scripts.
Per CONTRIBUTING.md, test flags should be minimal hooks that:
- Accept input, call the internal function, and output the result
- Do NOT contain test logic (assertions, validation, etc.)

Changes:
1. Added json_to<RebaseCommand> and json_to<RebaseLine> serialization
   so the -A flag can use json_to<std::vector<RebaseLine>> instead of
   60+ lines of manual JSON parsing.

2. Simplified -P flag to output JSON directly instead of formatted
   text with switch statements.

3. Added helper functions to run_test.sh for parsing rebase JSON output
   (rebase_line_count, rebase_command_at, rebase_index_at, rebase_role_at).

4. Updated test_rebase_parser.sh to use JSON output and helper functions.

The -E, -e, -R, -J, -B, and -T flags are already minimal hooks and
remain unchanged.
The -J flag was outputting 'valid' or 'discarded' instead of the raw
JSON result. This was test logic that should be in the test scripts.

Changed the -J flag to output the raw JSON result (pretty-printed) or
an error message. Updated the test script to use jq for JSON validation.

All other debug flags (-E, -e, -R, -P, -B, -A, -T) were already minimal
hooks and didn't need to be changed.
- Changed parse_json_safely to return std::expected<nlohmann::json, std::string>
- Updated all usages to handle the expected return type properly
- Simplified test flags in main.cpp to be minimal hooks without extra validation
- Removed try/catch blocks and explicit error handling from test flags
- All tests pass
- Add test for editing a message and removing all content (empty content)
- Add test for multiple consecutive edits to the same message (changes stack)
- Add test for editing messages with special characters
- Add test for very long message content
- Add test for editing assistant messages with tool_calls
- Add test for dropping all messages (empty history result)
- Add test for unicode characters in message content
- Add test for whitespace variations in rebase file
- Add test for comments and empty lines mixed with commands
- Add test for special characters and unicode in rebase content display
- Fix apply_rebase to preserve original content when only role is changed
- Add content_edited flag to RebaseLine to track explicit edits
Document the -b/--edit-history flag and rebase editor format.
Covers commands (pick/edit/drop), role editing, and common workflows.
As per issue #115, moved the History Editing documentation from README.md
to a separate docs/history-editing.md file. Added a short note with a link
to the new documentation in README.md.
This refactors the duplicated code for deserializing HarnessState from a file
into a reusable helper function. The same pattern was used in two places:
1. In the edit_history block
2. In the main flow when loading existing chat files

The helper function handles:
- Opening the file
- Reading file content
- Parsing JSON
- Converting to HarnessState
- Error reporting

This reduces code duplication and makes future modifications easier.
- Removed the errors vector from parse_rebase_file()
- Changed all error handling to return std::unexpected() immediately
- Removed the error aggregation code at the end of the function
- Updated test to verify first error is returned (not all errors)
- Added format_error() helper function to centralize line number error formatting
- Replaced all 11 occurrences of inline line number formatting with calls to format_error()
- No functional changes - all 33 tests pass
Add string_toul() function in utility.cpp that wraps std::stoul and returns
std::expected<unsigned long, std::string> instead of throwing exceptions.

Replace all uses of std::stoul with string_toul():
- src/main.cpp: 2 occurrences in test flags
- src/rebase_engine.cpp: 1 occurrence in parse_rebase_file()

The wrapper handles null input, empty input, and trailing characters,
converting any exceptions from std::stoul into error strings.
- Modified parse_rebase_file to skip header comments before '-- >8 --'
- Modified apply_rebase to drop messages whose indices are not mentioned
  in the parsed commands (implicit drop)
- Special case: if no commands are parsed (all lines deleted), keep all messages
- Updated test_implicit_pick to reflect new behavior
- Added test_deleting_lines_drops_messages and test_all_lines_deleted_keeps_all
- Updated test_delete_line_equals_drop to reflect new behavior
Split the monolithic test_edit_history.sh file into 21 individual
test scripts, each containing a single test. This follows the
project's testing guideline of having no more than one test per script.

Each test file contains its own helper functions (run_drharness,
get_output, reset_editor_state) and runs independently.

Updated CMakeLists.txt to register all 21 new test targets and
removed the old test_edit_history target.
Add create_history_json() helper function to run_test.sh and use it in
test_edit_prompt.sh to avoid manual JSON string construction. This follows
CONTRIBUTING.md guidelines that prohibit JSON handling in test files.
Split the monolithic test_edit_prompt.sh into 6 individual test files:
- test_edit_prompt_with_context.sh
- test_edit_prompt_first_message.sh
- test_edit_prompt_separator_placement.sh
- test_edit_prompt_multiline_context.sh
- test_edit_prompt_tool_role.sh
- test_edit_prompt_middle_message.sh

Each test is now its own script that can be run independently.
Add helper functions to run_test.sh for creating rebase lines and history
messages:
- create_rebase_line: creates a single rebase line JSON object
- create_rebase_lines: creates a JSON array of rebase lines
- create_message_json: creates a message JSON with optional fields
- create_messages_json: creates a JSON array of messages

Update test_rebase_apply.sh to use these helper functions instead of
manually constructing JSON strings, which violates CONTRIBUTING.md.
Replace hardcoded /tmp/drharness_output.txt with $TEST_TEMP_DIR/drharness_output.txt
in all test files. Tests should only write to subfiles/directories of the working
directory (which is already a temp directory created by run_test.sh).
- Consolidated dr_* functions into dr_field helper
- Refactored create_chat_file to use create_history_json
- Refactored create_rebase_lines to use create_rebase_line
- Created add_message_field helper for message field handling
- Created is_message_field helper for field name checking
- Refactored create_messages_json to use create_message_json
- Added local variable declarations for better scoping
- test_rebase_apply_pick_preserves.sh
- test_rebase_apply_drop_removes.sh
- test_rebase_apply_edit_updates_role.sh
- test_rebase_apply_edit_preserves_fields.sh
- test_rebase_apply_implicit_pick.sh
- test_rebase_apply_mixed_commands.sh
- test_rebase_apply_empty_lines_preserves.sh
- test_rebase_apply_lenient_orphaned_results.sh
- test_rebase_apply_edit_preserves_tool_calls.sh
- test_rebase_apply_deleting_lines_drops.sh
- test_rebase_apply_all_lines_deleted_keeps.sh

Removed the combined test_rebase_apply.sh file and updated CMakeLists.txt
to include the 11 new individual test files.
The tests have been split into individual test files and registered
in CMakeLists.txt. Removing the combined test file as it is no
longer needed.
Each test in tests/test_rebase_content.sh is now its own test script:
- test_rebase_content_basic_messages.sh
- test_rebase_content_multiline_content.sh
- test_rebase_content_tool_call_display.sh
- test_rebase_content_tool_message.sh
- test_rebase_content_empty_message.sh
- test_rebase_content_header_comments.sh
- test_rebase_content_multiple_tool_calls.sh
- test_rebase_content_message_indices.sh
- test_rebase_content_scissors_line.sh
- test_rebase_content_special_characters.sh
- test_rebase_content_unicode_characters.sh
- test_rebase_content_very_long_content.sh

All 83 tests pass.
Updated all 12 test_rebase_content_* test files to use the
create_messages_json helper function from run_test.sh instead of
manually creating JSON strings, following the pattern established
by test_rebase_apply_* tests.

This complies with CONTRIBUTING.md which states:
'If you do any json handling in the test files your changes will be rejected'

All 83 tests pass.
The file was split into 12 individual test files in a previous
commit, but the original combined file was not deleted. This
commit removes the now-obsolete test_rebase_content.sh file.
Each of the 17 tests from test_rebase_parser.sh is now in its own
test file, following the project convention of one test per script.
The add_message_field function now constructs tool call JSON internally
from simple parameters (id, name, arguments) instead of accepting raw
JSON strings. This makes the test helper functions more robust and
easier to use.

Changed:
- add_message_field: tool_call field now takes id, name, args and builds JSON
- create_message_json: updated to handle tool_call's 3-parameter syntax
- create_messages_json: updated to handle tool_call's 3-parameter syntax
- is_message_field: changed tool_calls to tool_call
- Updated all test files that used the old tool_calls raw JSON syntax
- Update ollama_client.cpp to use std::expected pattern for parse_json_safely
- Add missing 'index' field to tool_calls in add_message_field helper
- Add missing 'index' field to test_edit_history_assistant_with_tool_calls test data
Resolved merge conflicts:
- CMakeLists.txt: Added both rebase_engine/utility and base64_util sources
- src/harness.cpp: Combined Content type with proper error handling
- src/main.cpp: Merged includes, test flags, and CLI parsing
  - Renamed conflicting flags: -B→-Z (build_rebase), -T→-W (edit_prompt), -b→-K (base64_decode)
- tests/CMakeLists.txt: Added all test targets from both branches
- src/rebase_engine.cpp: Fixed to use get_content_string() for Content variant type
- Updated test files to use new flag names (-Z, -W, -K)
- Mark issue 169 as closed (merge completed)
- Unblock issue 93 (dependency on 169 satisfied)
Remove duplicate '-R' help line that was incorrectly merged with
inconsistent indentation.
darthrubik merged commit 975069c9d0 into master 2026-07-23 17:03:56 -05:00
darthrubik deleted branch feature/rebase 2026-07-23 17:03:56 -05:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: darthrubik/drharness#44