- Default `p -- <cmd>` now streams logs via SSH (like tail -f) - Ctrl+C detaches from stream; job keeps running on worker - Add `-d/--detach` flag to start job without streaming - Remove `p attach` command (use `p logs -f` instead) - Remove p-agent daemon; jobs launched via nohup over SSH - Simplify worker requirements: only rsync needed (no tmux, no agent) - Jobs managed via ad-hoc SSH: kill $(cat pid), tail -f output.log
This commit is contained in:
@@ -1,14 +1,15 @@
|
|||||||
# `p` — push jobs to worker
|
# `p` — push jobs to worker
|
||||||
|
|
||||||
A small Rust CLI utility to push command-line jobs to remote worker machines,
|
A small Rust CLI utility to push command-line jobs to remote worker machines,
|
||||||
with directory sync, job management, and attach/detach support.
|
with directory sync, job management, and log streaming.
|
||||||
|
|
||||||
## Motivation
|
## Motivation
|
||||||
|
|
||||||
The common developer workflow of "run this build/test/script on a more powerful
|
The common developer workflow of "run this build/test/script on a more powerful
|
||||||
remote machine" currently requires manually chaining `rsync`, `ssh`, and `tmux`.
|
remote machine" currently requires manually chaining `rsync` and `ssh` with
|
||||||
|
a way to keep the job alive in the background (e.g. `nohup`, `tmux`).
|
||||||
`p` wraps that entire flow into a single ergonomic command, while adding proper
|
`p` wraps that entire flow into a single ergonomic command, while adding proper
|
||||||
job tracking, log capture, and attach/detach mechanics.
|
job tracking and log capture.
|
||||||
|
|
||||||
## Core Concepts
|
## Core Concepts
|
||||||
|
|
||||||
@@ -26,13 +27,6 @@ directory. Each job has:
|
|||||||
- Start time, end time, exit code
|
- Start time, end time, exit code
|
||||||
- Captured output log
|
- Captured output log
|
||||||
|
|
||||||
### p-agent
|
|
||||||
A small Rust (?) binary, automatically uploaded and started by `p` on first use
|
|
||||||
of a worker. It manages job lifecycle, log capture, and status tracking on
|
|
||||||
the worker side. The user never manually installs or configures it.
|
|
||||||
`p` checks the agent version on each connection and re-uploads if outdated.
|
|
||||||
Communication happens over SSH port forwarding — no extra open ports needed.
|
|
||||||
|
|
||||||
## CLI Reference
|
## CLI Reference
|
||||||
|
|
||||||
### Running jobs
|
### Running jobs
|
||||||
@@ -41,14 +35,19 @@ Communication happens over SSH port forwarding — no extra open ports needed.
|
|||||||
p -- <command>
|
p -- <command>
|
||||||
```
|
```
|
||||||
Sync the current directory to the default worker and run `<command>` on it.
|
Sync the current directory to the default worker and run `<command>` on it.
|
||||||
Attaches to the job's tmux session immediately. `Ctrl+B D` detaches without
|
Streams the job's output directly to the terminal (like `tail -f`). This feels
|
||||||
killing the job. `Ctrl+C` sends SIGINT to the running process (standard behavior).
|
like running the command locally.
|
||||||
|
|
||||||
When the job finishes, the session stays open and displays:
|
- `Ctrl+C` detaches from the log stream — the job keeps running on the worker.
|
||||||
|
Use `p logs -f <job-id>` to resume watching.
|
||||||
|
- Use `p stop <job-id>` to kill a running job.
|
||||||
|
- If the network connection drops, the job keeps running on the worker.
|
||||||
|
Use `p logs -f <job-id>` to resume watching.
|
||||||
|
|
||||||
|
When the job finishes, `p` prints the exit code and exits:
|
||||||
```
|
```
|
||||||
--- Job done [exit 0]. Press any key to detach. ---
|
[Job done: exit 0]
|
||||||
```
|
```
|
||||||
This lets the user read final output before returning to their shell.
|
|
||||||
|
|
||||||
```
|
```
|
||||||
p <worker> -- <command>
|
p <worker> -- <command>
|
||||||
@@ -59,7 +58,14 @@ Same, but targets a specific named worker.
|
|||||||
p [-n | --no-sync] -- <command>
|
p [-n | --no-sync] -- <command>
|
||||||
```
|
```
|
||||||
Run `<command>` on the worker without syncing the current directory first.
|
Run `<command>` on the worker without syncing the current directory first.
|
||||||
Useful for commands that need no local files (e.g. `p -n -- htop`).
|
Useful for commands that need no local files.
|
||||||
|
|
||||||
|
```
|
||||||
|
p [-d | --detach] -- <command>
|
||||||
|
```
|
||||||
|
Run `<command>` and immediately detach — do not stream output to the terminal.
|
||||||
|
The job starts on the worker and `p` prints the job ID. Useful for fire-and-forget
|
||||||
|
jobs. Use `p logs -f <job-id>` to watch later.
|
||||||
|
|
||||||
### Job management
|
### Job management
|
||||||
|
|
||||||
@@ -71,19 +77,12 @@ completed jobs (done, failed, stopped).
|
|||||||
Shows: ID (short), worker, original CWD, command, status, duration.
|
Shows: ID (short), worker, original CWD, command, status, duration.
|
||||||
Style inspired by `docker ps` / `lxc list`.
|
Style inspired by `docker ps` / `lxc list`.
|
||||||
|
|
||||||
```
|
|
||||||
p attach <job-id>
|
|
||||||
```
|
|
||||||
Re-attach to the tmux session of a running job. Supports partial IDs.
|
|
||||||
Behaves identically to the initial attach: `Ctrl+B D` detaches, and if the job
|
|
||||||
has already finished the "press any key" screen is shown.
|
|
||||||
Only works on **running** jobs. For finished jobs, use `p logs`.
|
|
||||||
|
|
||||||
```
|
```
|
||||||
p logs <job-id>
|
p logs <job-id>
|
||||||
```
|
```
|
||||||
Print the captured output of a job (running or finished). Supports `-f` to
|
Print the captured output of a job (running or finished). Supports `-f` to
|
||||||
follow a running job's output without attaching to its TTY.
|
follow a running job's output in real-time. `Ctrl+C` detaches without stopping
|
||||||
|
the job.
|
||||||
|
|
||||||
```
|
```
|
||||||
p stop <job-id>
|
p stop <job-id>
|
||||||
@@ -148,62 +147,40 @@ Set the default worker.
|
|||||||
- No automatic sync-back after job completion. Use `p pull` to retrieve
|
- No automatic sync-back after job completion. Use `p pull` to retrieve
|
||||||
specific artifacts.
|
specific artifacts.
|
||||||
|
|
||||||
## Attach / Detach Mechanics
|
## Execution Model
|
||||||
|
|
||||||
Jobs run inside a `tmux` session on the worker. `p` attaches to the session
|
No persistent agent daemon is needed. Jobs are launched and managed via
|
||||||
immediately after starting the job.
|
ad-hoc SSH commands:
|
||||||
|
|
||||||
### Status bar
|
1. `p -- <command>` syncs the directory, then runs via SSH:
|
||||||
The tmux session has a custom status bar showing:
|
|
||||||
```
|
```
|
||||||
p-<short-id> beefy make [running] 0:02:14
|
nohup sh -c '<command> 2>&1 | tee output.log; echo $? > exitcode' & echo $! > pid
|
||||||
```
|
```
|
||||||
Fields: job short-ID, worker name, command (truncated), status, elapsed time.
|
2. The client streams `output.log` in real-time over a separate SSH connection.
|
||||||
|
3. `Ctrl+C` closes the SSH stream — the job keeps running.
|
||||||
|
4. `p stop <job-id>` runs `kill $(cat pid)` over SSH.
|
||||||
|
5. `p logs -f <job-id>` tails the log file over SSH.
|
||||||
|
6. `p ls` reads the local job DB and SSH-polls to reconcile state when needed.
|
||||||
|
|
||||||
### Key bindings while attached
|
> **Worker requirements:** `rsync` must be available on the worker.
|
||||||
| Key | Effect |
|
|
||||||
|---|---|
|
|
||||||
| `Ctrl+B D` | Detach from session. Job keeps running. |
|
|
||||||
| `Ctrl+C` | Sends SIGINT to the foreground process (standard terminal behavior). |
|
|
||||||
|
|
||||||
### On job completion
|
|
||||||
When the job's process exits, `run.sh` writes the exit code and then displays:
|
|
||||||
```
|
|
||||||
--- Job done [exit 0]. Press any key to detach. ---
|
|
||||||
```
|
|
||||||
The tmux session stays open (`remain-on-exit on` for the window) so the user
|
|
||||||
can scroll through final output. Pressing any key detaches the client and
|
|
||||||
returns to the local shell. `p` then reads the exit code and prints a summary.
|
|
||||||
|
|
||||||
### `p attach` on a finished job
|
|
||||||
If the job has already finished and the tmux session is still open (user has
|
|
||||||
not yet pressed a key), `p attach` reconnects to the "press any key" screen.
|
|
||||||
Once the key is pressed, the session closes. For a fully-closed session, use
|
|
||||||
`p logs` instead.
|
|
||||||
|
|
||||||
> **Worker requirements:** `tmux` and `rsync` must be available on the worker
|
|
||||||
> (standard on most Linux systems). The `p-agent` binary is auto-uploaded by `p`.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Job Status & Notification
|
## Job Status & Tracking
|
||||||
|
|
||||||
The **p-agent** runs as a lightweight background process on the worker
|
The client maintains a local job database (`~/.local/share/p/jobs/<uuid>.json`).
|
||||||
(started automatically, not a system service). It:
|
`p ls` reads from this local store for fast output.
|
||||||
|
|
||||||
- Manages job launch and tmux session creation
|
### State reconciliation
|
||||||
- Tees output to `output.log`
|
When a job is running, the client periodically checks if `exitcode` exists on
|
||||||
- Writes `exitcode` on completion
|
the worker. If the client was offline or the connection dropped, the next
|
||||||
- Notifies the client over the SSH reverse tunnel when a job finishes
|
`p ls` SSH-polls workers to reconcile state. Jobs with unknown status are
|
||||||
|
marked accordingly.
|
||||||
|
|
||||||
The client maintains a local job database (`~/.local/share/p/jobs/<uuid>.json`)
|
### Connection drops during streaming
|
||||||
mirroring job state. `p ls` reads from this local store (fast, no SSH),
|
If the SSH connection drops while `p` is streaming output, `p` exits with an
|
||||||
updated in real time while attached, and via agent notifications otherwise.
|
error message showing the job ID. The job continues running on the worker.
|
||||||
|
Resume watching with `p logs -f <job-id>`.
|
||||||
### Degraded mode (agent unreachable / client was offline)
|
|
||||||
If the client missed a completion notification, `p ls` marks affected jobs as
|
|
||||||
`unknown`. The next `p ls` SSH-polls all workers with known-running jobs to
|
|
||||||
reconcile state.
|
|
||||||
|
|
||||||
## Worker-side Layout
|
## Worker-side Layout
|
||||||
|
|
||||||
@@ -211,8 +188,6 @@ All data lives under `~/.p/` on the worker (no root access required).
|
|||||||
|
|
||||||
```
|
```
|
||||||
~/.p/
|
~/.p/
|
||||||
bin/
|
|
||||||
p-agent # auto-uploaded by p, versioned
|
|
||||||
jobs/
|
jobs/
|
||||||
<uuid>/
|
<uuid>/
|
||||||
cmd # command string
|
cmd # command string
|
||||||
@@ -221,7 +196,8 @@ All data lives under `~/.p/` on the worker (no root access required).
|
|||||||
started_at # unix timestamp
|
started_at # unix timestamp
|
||||||
output.log # combined stdout+stderr, always captured
|
output.log # combined stdout+stderr, always captured
|
||||||
exitcode # written on completion; absent = still running
|
exitcode # written on completion; absent = still running
|
||||||
tmux_session # tmux session name (e.g. "p-<short-uuid>")
|
pid # process ID of the running job
|
||||||
|
|
||||||
workdirs/
|
workdirs/
|
||||||
<uuid>/ # rsync'd copy of client CWD for this job
|
<uuid>/ # rsync'd copy of client CWD for this job
|
||||||
```
|
```
|
||||||
@@ -252,23 +228,9 @@ b004f123 cloud ~/scripts ./bench.sh done [1] 0:00:47
|
|||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
- **Worker arch detection**: `p-agent` must be compiled for the worker's
|
|
||||||
architecture. Options: (a) ship common targets and detect via SSH, (b)
|
|
||||||
compile on the worker if a Rust toolchain is present, (c) require user to
|
|
||||||
specify arch in worker config.
|
|
||||||
|
|
||||||
Maybe we could also implement the agent core in the form of a shell script?
|
|
||||||
At least the entry point, which could do some detection and install or something.
|
|
||||||
We will see on implementation what works... Small rust binary seems nice,
|
|
||||||
but we want support for amd64, aarch64 and riscv64.
|
|
||||||
|
|
||||||
- **Multiple jobs from the same CWD**: each gets its own `workdirs/<uuid>/`,
|
- **Multiple jobs from the same CWD**: each gets its own `workdirs/<uuid>/`,
|
||||||
so they're fully isolated. This may use significant disk space — `p rm`
|
so they're fully isolated. This may use significant disk space — `p rm`
|
||||||
should prompt to clean up.
|
should prompt to clean up.
|
||||||
|
|
||||||
- **Non-Linux workers**: tmux availability and path conventions may differ on
|
- **Non-Linux workers**: path conventions may differ on macOS workers. Out of
|
||||||
macOS workers. Out of scope for now.
|
scope for now.
|
||||||
|
|
||||||
- **Ctrl+C → detach** (future): it would be nicer if Ctrl+C detached the
|
|
||||||
session instead of sending SIGINT to the job, matching the spirit of the
|
|
||||||
tool. This requires per-session tmux key table configuration and is deferred.
|
|
||||||
|
|||||||
Reference in New Issue
Block a user