Troubleshooting

Common Issues

Here are solutions to common issues you might encounter with the Keystash Agent:

Agent Not Starting

Check the agent status using systemd:
```
systemctl status keystash
```
Review the logs:
```
journalctl -u keystash
```
Verify configuration file permissions:
```
ls -l /etc/keystash/keystash.conf
# Should show: -rw------- 1 root root
```
Note

The agent enforces 0600 permissions on /etc/keystash/keystash.conf and will refuse to start if the file is left more permissive and cannot be repaired. See Configuration File Basics.

Authentication Failures

Check your configuration:

Verify account_id and deployment_secret in your config file
Ensure network connectivity to server-api.keystash.io (or your configured endpoint)
Check proxy settings if using a proxy

User Synchronization Issues

Verify permissions:

Check if agent can write to /etc/passwd
Verify permissions on user home directories
Look for conflicts with local user management tools

SSH Key Problems

Check file permissions:

~/.ssh directories should be 700
authorized_keys files should be 600
Verify SELinux/AppArmor aren't blocking access

Two Factor Authentication

As of v1.6, Two Factor Authentication is handled entirely by the bundled pam_keystash.so module — there is no longer a separate google-authenticator binary or libpam-google-authenticator package to check.

The enforcing PAM line in /etc/pam.d/sshd is auth required pam_keystash.so socket_path=/var/run/keystash/auth.sock fallback=deny attempts=3. The agent only writes this line once it has confirmed the module is present on disk and its IPC socket is reachable, so it will never leave you with a deny-all SSH configuration.
The module's fallback= argument controls what happens when it cannot reach the agent: fallback=deny (the default) refuses the login, while fallback=allow_with_warning permits it and logs a warning. The attempts= argument limits how many code attempts a user gets (default 3).
TOTP secrets are held in memory by the agent and served to the module over IPC, so there is no separate authenticator binary to configure. The legacy ~/.ssh/two_factor_auth_settings file is no longer written on any distribution; if an older agent left one behind, the agent deletes it during the transition.

Logging

The Keystash Agent involves two separate kinds of "logging" — keep them distinct when troubleshooting:

Agent diagnostic logging - the agent's own operational output (sync results, errors, and the startup endpoint summary). This is what you read with journalctl, enable in more detail with --debug, and redirect with --log-target.
SSH Connection Logging - the audit feature that records SSH session events and surfaces them in the Keystash app. This is configured per Server Group and is documented on the SSH Connection Logging page.

To work with the agent's diagnostic logging:

Enable debug logging:

/opt/keystash/keystash-key-manager.bin --debug

View logs:
Systemd logs: journalctl -u keystash
Upgrade-specific log: /opt/keystash/keystash_update.log

SSH Connection Logging Not Appearing

If SSH sessions are not showing up under Logs > SSH Sessions in the Keystash app, work through this checklist on an affected server:

Confirm the Server Group toggle. SSH Connection Logging is enabled per Server Group. Check that "Enable SSH Connection Logging" is on for the group this server belongs to — see Enabling SSH Connection Logging.
Confirm the agent version and sync state. The feature requires agent v1.6 or newer, and the agent needs to have synced since the toggle was enabled (around 60 seconds):
```
systemctl status keystash
journalctl -u keystash --since "5 minutes ago"
```
Confirm the PAM line is present. When logging is enabled, the agent adds a session line to the SSH PAM stack:
```
grep pam_keystash.so /etc/pam.d/sshd
# Expect a line: session optional pam_keystash.so
```
Confirm the IPC socket exists. The PAM module talks to the agent over a local Unix socket. If the socket is missing, the agent is not running:
```
ls -l /var/run/keystash/auth.sock
```
Confirm the spool is draining. Events are spooled to disk before upload. The directory should be filling as sessions occur and draining on each sync tick:
```
ls /var/lib/keystash/session-events/
```
Grep the agent log for recorded events. Each captured event produces a log line carrying its session ID. This line is emitted at debug level, so run the agent with --debug (or enable debug logging) to see it:
```
journalctl -u keystash | grep 'connection_log recorded'
```
A session=- value means the event was dropped before a session could be resolved, or forwarding was disabled.

Additional Help

Note

If you're experiencing issues not covered in this documentation, please reach out to Keystash Support.