Data Formats

Technical specifications for xyOps data exchange formats including XYPDF, XYBK, and XYWP

xyOps defines three standardized data formats for portability, backup/restore, and inter-process communication.

XYPDF - Portable Data Format

The xyOps Portable Data Format (XYPDF) enables export, transfer, and import of individual objects between xyOps installations.

• Format: JSON (plain text .json or gzip .json.gz)
• Version: 1.1
• Use Case: Sharing plugins, workflows, alerts, and configurations

File Structure

XYPDF files contain a wrapper with metadata and an array of typed items:

{
  "type": "xypdf",
  "version": "1.0",
  "xyops": "1.0.0",
  "description": "xyOps Portable Data",
  "items": [
    {
      "type": "web_hook",
      "data": { /* actual object */ }
    }
  ]
}

type · string · required

File format identifier, must be xypdf.

version · string · required

File format version, currently 1.0.

xyops · string

Minimum supported xyOps version in semver format (added in XYPDF v1.1).

description · string

Human-readable description, typically xyOps Portable Data.

items · array · required

Array of objects to import. Each has type and data properties.

Supported Object Types

XYPDF can export/import the following object types:

Object Type	Type ID	Notes
Alert	alert	Alert definitions
API Key	api_key	API access keys
Bucket	bucket	Metadata only (not files)
Category	category	Job categories
Channel	channel	Notification channels
Event	event	Events and workflows
Group	group	Server groups
Monitor	monitor	Monitor definitions
Plugin	plugin	Plugin definitions
Role	role	User roles
Tag	tag	Tag definitions
WebHook	web_hook	Webhook configurations

See for object schemas.

Export Example

WebHook Export

{
  "type": "xypdf",
  "version": "1.0",
  "xyops": "1.0.0",
  "description": "xyOps Portable Data",
  "items": [
    {
      "type": "web_hook",
      "data": {
        "id": "wmb6q7bh3hy",
        "title": "Discord",
        "enabled": true,
        "url": "https://discord.com/api/webhooks/123456789/abcdefghi",
        "method": "POST",
        "headers": [
          {
            "name": "Content-Type",
            "value": "application/json"
          }
        ],
        "body": "{\n\t\"content\": \"{{text}}\"\n}",
        "timeout": 30,
        "retries": 0,
        "notes": "Posts to #general channel"
      }
    }
  ]
}

Import Behavior

When importing XYPDF files:

1. Security review

   User must review the entire file contents in a popup dialog before confirming.

   Never import XYPDF files from untrusted sources. They may contain malicious code in plugins or workflow scripts.

2. Conflict detection

   If objects with matching IDs exist, user is warned and must confirm replacement.

3. Import execution

   Objects are created or replaced. Dependencies (plugins, events) are automatically included when exporting workflows.

XYBK - Backup Format

The xyOps Backup Format (XYBK) supports bulk export/import of entire databases, lists, and files.

• Format: NDJSON (newline-delimited JSON), optionally gzipped
• Version: 1.0
• Use Case: Full system backups, migrations, disaster recovery

File Structure

XYBK files are line-oriented with three types of lines:

1. Comments: Lines starting with # (ignored)
2. Blank lines: Whitespace-only (ignored)
3. Records: JSON objects (one per line)

# xyOps Data Export v1.0
# Hostname: production.xyops.io
# Date/Time: Tue Nov 18 2025 12:01:27 GMT-0800
# Format: NDJSON

{"cmd":"listDelete","args":["global/alerts",false]}
{"key":"global/alerts","value":{"page_size":100,"length":5,"type":"list"}}
{"index":"tickets","id":"tmhzbmbagig","record":{"subject":"High CPU Alert"}}

Record Types

Storage Put

Write key/value pairs to storage:

{"key": "users/alice", "value": {"username": "alice", "email": "[e-mail]"}}

• For binary keys, value is Base64-encoded
• For JSON keys, value is the object itself

Storage Command

Execute storage API methods:

{"cmd": "listDelete", "args": ["global/alerts", false]}

Used to prepare state (e.g., delete lists before re-creating).

Database Record

Insert records into Unbase indexes:

{"index": "tickets", "id": "tmhzbmbagig", "record": {"subject": "..."}}

Semantics: create or replace by ID.

Data Selection

Exports can include:

Lists

Standard xyOps configuration lists:

• alerts
• api_keys
• buckets
• categories
• channels
• events
• groups
• monitors
• plugins
• roles
• secrets
• tags
• users
• web_hooks

Choosing users also triggers user account data export.

Database Indexes

Full Unbase indexes with optional query filters:

• jobs - Job execution history
• alerts - Alert events
• tickets - Ticket system
• servers - Server registration
• snapshots - System snapshots
• activity - Activity log

Example filter: jobs?result=error&created:>2024-01-01

Extras

Optional large datasets:

• job_files - Job file uploads
• job_logs - Compressed job logs
• bucket_files - Bucket file payloads
• ticket_files - Ticket attachments
• monitor_data - Monitor time-series
• stat_data - Statistics data
• user_avatars - User avatar images

Example Export

# xyOps Data Export v1.0
# Hostname: joemax.xyops.io
# Date/Time: Tue Nov 18 2025 12:01:27 GMT-0800
# Format: NDJSON

# List: global/alerts
{"cmd":"listDelete","args":["global/alerts",false]}
{"key":"global/alerts","value":{"page_size":100,"first_page":0,"last_page":0,"length":5,"type":"list"}}
{"key":"global/alerts/0","value":{"type":"list_page","items":[{"id":"load_avg_high","title":"High CPU Load"}]}}

# Database Index: tickets (*)
{"index":"tickets","id":"tmhzbmbagig","record":{"subject":"Alert: High Active Jobs","status":"open"}}

Security Characteristics

XYBK exports contain sensitive data in specific formats for security:

• API Keys: Only salted SHA-256 hashes exported (no plaintext keys)
• Secrets: Encrypted blobs in Base64 (no plaintext)
• Passwords: Salted bcrypt hashes only

Import Process

1. Stream parsing

   File is processed line-by-line. Comments and blank lines are ignored.

2. Record execution

   • key/value → storage.put(key, value)
   • cmd/args → Execute storage method
   • index/id/record → unbase.insert(index, id, record)
3. Error handling

   Up to 100 errors collected. Import continues past non-fatal errors.

XYWP - Wire Protocol

The xyOps Wire Protocol (XYWP) defines communication between xyOps and plugins via STDIO pipes.

• Format: NDJSON over STDIN/STDOUT
• Version: 1.0
• Use Case: Plugin execution, inter-process communication

Protocol Properties

All XYWP messages include:

xy · number · required

Wire protocol version, always 1.

Request Messages

xyOps sends requests to plugins:

{
  "xy": 1,
  "type": "event",
  "id": "jmhzaot10tm",
  "event": "emi11ejdlde",
  "plugin": "pmi11dqsxcy",
  "server": "smf4j79snhe",
  "now": 1763256572.024
}

type · string

Message type indicating intent (e.g., event for job execution).

Additional properties depend on message type. See Job data structure.

Response Messages

Plugins send responses back to xyOps:

Progress Update

Intermediate updates without code property:

{"xy": 1, "progress": 0.5}
{"xy": 1, "progress": 0.75, "message": "Processing..."}

Plugin continues running after progress updates.

Success Response

Final response with code: 0 indicates success:

{"xy": 1, "code": 0}
{"xy": 1, "code": 0, "description": "Task completed"}

Plugin exits after final response.

Error Response

Non-zero code indicates error:

{"xy": 1, "code": 999, "description": "Failed to connect to database."}
{"xy": 1, "code": 1, "description": "Validation failed"}

Plugin exits after error response.

Passthrough JSON

JSON without the xy property (or with xy != 1) is treated as plain text and logged:

{"level": "info", "message": "Starting backup..."}

This allows plugins to emit structured logs that aren’t XYWP messages.

Communication Flow

1. Plugin Launch

   xyOps spawns plugin as subprocess with STDIO pipes connected.

2. Request Sent

   xyOps writes XYWP request to plugin’s STDIN (one line, newline-delimited).

3. Progress Updates

   Plugin optionally writes progress updates to STDOUT (XYWP format).

4. Final Response

   Plugin writes final response with code property to STDOUT.

5. Plugin Exit

   Plugin exits. xyOps reads exit code and captures any STDERR output.

Only STDIN and STDOUT are part of the protocol. STDERR is captured as plain text and displayed to users on errors.

Example Plugin (Python)

#!/usr/bin/env python3
import json
import sys

# Read request from STDIN
request = json.loads(sys.stdin.readline())

if request.get('xy') != 1:
    print('{"xy": 1, "code": 1, "description": "Invalid protocol version"}', flush=True)
    sys.exit(1)

# Send progress
print('{"xy": 1, "progress": 0.5}', flush=True)

# Do work...

# Send success
print('{"xy": 1, "code": 0, "description": "Task completed"}', flush=True)
sys.exit(0)

References

• Unbase Database System
• pixl-server-storage
• NDJSON Specification