pgs: document http-pass acl type in acl help and validation error (#224)

The acl command already accepts --type http-pass (password login form),
but the -type flag help and the validation error omitted it. Also note
that for http-pass the -acl value is the password.

feat(metrics): label http requests and bytes with a resource

fix: correct typos in error and log messages

feat(metrics): export server info with nodename and commit

feat(metrics): let servers outside net/http record user metrics

feat(caddy): require a token for metrics routes

feat(metrics): count http requests and ssh sessions per user

perf(db): add missing indexes, drop unused ones, trim queries

fix(analytics): stop counting mobile visitors as desktop

fix(ssh): label access logs with the public key's name

fix(router): look up custom domain records as fully qualified names

fix(pgs): preserve request URL through password login

feat(pastes,prose): print help instead of the deprecation notice

fix(pico): print help for unknown commands and sessions without a pty

fix(pssh): apply pty-req before running the session's command

chore: remove gha

fix(tui): stricter bounds checking for tui height when drawing

docs(prose): invite only

docs: readme

fix(rsync): accept --relative on uploads

fix(pici): golangci-lint

fix(pgs): reject directories during GetObject calls

We were previously relying on the etag computation via io.Copy to implicitly
reject directories.

fix(pgs): serve bin files as binary content (#226)

test(pgs): drop the fixed prometheus port from the test servers

test(pipe): drop the fixed prometheus port from the test server

ci: build images with docker bake and cache go builds across runs

ci: run on ubuntu 26.04 with a test timeout and fix slow rsync and scp tests

ci: restore github actions

feat(pico): check where a domain points in dns-check and pin pico-ssh dns

Merge pull request #227 from picosh/am/decouple-apps

Decouple apps for more standalone dep graphs

refactor(postfs): move post file ops out of filehandlers and the router into prose

refactor(db): share the auth lookups and feature check across apps

fix(prose,pipe): serve a post named raw and bound pipe-web connection setup

fix(pipe): send an exit status when commands end

fix(pipe): end pipe-web sessions with their requests

fix(pssh): end a session's command when the client closes it

refactor(apps): give each app its own config

fix(pubsub): deliver a publisher's last message before closing subscribers

refactor(router): serve apps with http.ServeMux and typed handlers

refactor(apps): give each app its own uploader and db queries

test: make expiry and pgs url tests independent of local timezone and env

feat(uploads): only show storage usage once a user reaches their quota

fix(pgs): copy a single file into a project that doesn't exist yet

Merge pull request #225 from picosh/am/rsync-protocol

Rsync rewrite and perf updates

feat(filehandlers): rename posts in place over sftp

feat(send): support rsync merge filters and sftp renames

fix(tui): show usage stats for users without pages projects

feat(features): honor standalone flags for every service

feat(invites): gate pastes and let invited users invite

feat(pastes): readd pastes service

feat(storage): remove dir marker

fix(rsync): ensure no leaked routines on failed upload

feat(pico): add dns-check and speed up pgs serving

perf(pgs): speed up uploads and fix quota accounting

feat(rsync): implement the rsync protocol natively

chore: set log limits for all services

refactor(prose): use ul+li for lxt rendering

fix(prose): safari text resizing pre blocks

refactor(prose): remove unused css from read

style(prose): read.css file

chore(prose): remove with_styles and layout front-matter fields

BREAKING CHANGE: these fields will no longer work for prose blogs

style(prose): minimal design ethos

Our previous prose blog design hasn't changed since we initially released prose.

The previous design was minimalistic but it still made some assumptions about 
how a blog should look.  In order to support customization we added support
for users to upload `_styles.css` and front-matter fields like `with_styles`
to provide full customization.

The main thesis for this change was trying to answer the question: what if blogs
came without styles by default and users are required to theme it themselves?
This would make it so we are not in control of the overall theme of the blog and
provides full customization for the end-user.

As we were experimenting with this design we wanted to include a very minimal
css reset so the default design is consistent across browsers.  We are also
implemented a few rules with this css reset for best practices.

This is our new design ethos:
- Center content on the screen
- 70ch max width
- Headers should be anchored to the content they are heading

That's it.  The rest of the reset file is for consistency across browsers.  With
this change we are also disabling the `with_styles` field to encourage people to
preserve this new design ethos.

Finally as part of this change we restructured the html tree to better adhere
to semantic meaning of those html entities.  We were already mostly there but
some of the headers and footers were restructured slightly.

fix: ci

Update deps

chore: remove dead code

chore: deadcode tool

chore: remove pastes service

We are going to keep pastes running but we want to remove it from our git tree
since it doesn't receive updates at this point and is mostly clutter.

style(prose): design tweaks

fix(pico): skip invite by when user is nil

refactor(prose): trending discovery feed

This changes the discovery feed to only show users that have prose access after
the invite system changes.

It also captures all post analytics for all prose users so we can use it for
the discovery feed ranking system.

fix: test

refactor(plus): 10GB -> 5GB

We only have a single user that's using more than 5GB so we are scoping the
allottment based on real-world usage.  This will also help us maintain healthy
services by clipping whale activity and preventing using pgs as a generic obj
storage system.

refactor: invite system for pgs and prose

This creates an invite system in order to be granted access to prose and pgs
services.

The goal here is to prevent spam.

Only pico+ users or users invited by someone else are allowed to invite people.

chore: reduce log retention

chore: rename pico.sh to pici.sh

fix: quote password to special chars dont corrupt line

chore(script): cert migration to valkey

refactor: use valkey instead of redis

refactor(pgs): use redis for central tls cert storage

chore: bump senpai to 0.5.0

fix(caddy): revert postgres storage backend

We are reciving reports of issues loading sites and our postgresql server
is struggling to keep up with load

fix(pipe): flaky test

fix(pgs): better pico+ error message when uploading

fix(caddy): pgs _metric for ash header_up

fix(caddy): subdomain for nue/ash pgs

chore: add cdn to pgs-caddy

chore(tui): check for pgs feature flag

docs: agent

refactor: remove github actions

chore(pgs): check for pgs feature flag

fix(pipe): cleanly terminate pipe sessions when peer disconnects

Previously, when one party in a bidirectional `pipe` disconnected, the
broker treated the remaining client as an active publisher (because its
direction is `InputOutput`), leaving the remaining client hanging in a
half-closed state with frozen terminal input.

- broker: unblock `Connect()` immediately when `client.Done` closes and
  trigger `client.Cleanup()` on read/write errors
- client: close underlying `ReadWriter` in `Cleanup()` if it implements `io.Closer`
- pipe/cli: implement `Close()` on `throttledMonitorRW`

chore(ci): better default zmx session prefix

fix(httpscache): flaky test

chore(pgs): uploader tests confirming pico+ only

refactor(pgs): validate pico+ before uploading so we don't create empty projects

fix(caddy): add postgres storage backend for distributed ACME challenges

This updates the Caddy Dockerfile to compile with the
`github.com/yroc92/postgres-storage` module and configures `Caddyfile.pgs` to
use it via the existing `DATABASE_URL` environment variable.

Previously, Caddy instances on independent nodes (e.g., US and EU) used their
own local filesystem for ACME storage. This caused on-demand TLS to hang or time
out for custom domains when Let's Encrypt's ACME verification challenge was
randomly routed by DNS to a node that didn't initiate the request.

By using a central Postgres database for storage, all Caddy nodes now share
certificate state and can seamlessly answer ACME challenges initiated by any
other node in the cluster.

chore(register): add some logging around blocking ips

chore: add more tests and ensure block and keepalive work with wildcards

feat(pipe): subscribe to wildcard topics

Our pipe service now supports wildcard topics `ssh pipe sub metric-drain*`.

This allows users to publish to multiple topics and have them drain into a
single subscriber.

ssh pipe pub metric-drain-x
ssh pipe pub metric-drain-y
ssh pipe sub "metric-drain*"

docs: rm bd from agents file

fix(pgs): correctly set storage max in uploader

docs(runbooks): delete account

fix: scp <file> pgs.sh:/<project> works

docs: runbook

fix(prose): use correct filepath for prose images

fix(pgs): missing cfg

revert(pgs): add file max logic

chore(pgs): service now requires pico+

chore: ban ip runbook

fix: missing pico-ssh in ci script

chore: update pico.sh script

feat: block account creation by ip

fix: mime type case statement fallthrough

fix(rsync): properly delete `._pico_keep_dir` first so os.Remove suceeds

fix(pgs): broken url for imgproxy integration

fix(pgs): never cache http-pass responses

Password-protected (http-pass) projects could be read without the
password. The shared cache keys entries on subdomain+method+uri with no
auth component, and only declines to store responses marked private or
no-store. http-pass assets were served as 200 with
`cache-control: max-age=60, s-maxage=600, must-revalidate`, so the first
authenticated request populated the cache and every subsequent
unauthenticated visitor got a cache hit that bypassed the password gate
for the duration of the TTL.

Mark http-pass asset responses `private, no-store` so the shared cache
refuses to store them. The override is applied after the user `_headers`
are merged, so a project's own cache-control cannot re-enable caching of
protected content.

Adds a regression test asserting an http-pass response is served
non-cacheable even when a `_headers` file requests aggressive caching.

fix: strip debug from linker

refactor: move go-rsync-receiver into pico monorepo

fix: require testcontainers for pico.sh ci

fix(ci): cleanup

chore(ci): cleanup

docs(pgs): help cmd for forms

chore: pico.sh ci script

fix(pgs): _headers should override default cache-control

Closes: https://github.com/picosh/pico/issues/215

feat(tui): analytics now displays device breakdown

You can see mobile vs desktop

docs: runbooks

fix: script

chore: script to find orphaned buckets

fix(auth): mime type can include charset

e.g. text/html; charset=utf-8

chore(visits): new indexes and parallel queries

chore: remove dev artifacts

refactor(visits): aggregate visit tables

Previously we had an analytics_visits table that held all the raw visit data.
This mean our analytics UI was constantly executing queries across 18 mil records
which was extremely slow.  This change aggregates those analytics every month
and then deletes all the raw data.

We keep 2 months worth of raw data and then everything else gets pushed into
the aggregate tables.

chore: script reap edits

fix: reap should look skip anyone that has pico+

chore: delete old analytics

chore: sql query to reap unused accounts

References: https://pico.prose.sh/ann-036-reap-inactive-accounts

fix(httpcache): cdn accept encoding

Two bugs combined to produce the broken behavior:

Bug 1: CDN forwarded Accept-Encoding to upstream (cmd/pgs/cdn/main.go)

proxyServe.ServeHTTP cloned the incoming request (including its Accept-Encoding: zstd header sent by Caddy) and forwarded it to ash.pgs.sh. The upstream responded with a zstd-compressed body + content-encoding: zstd. The CDN then cached that compressed blob. Caddy (sitting in front of the CDN) can't transcode
zstd→nothing, so clients received raw zstd bytes named .xml.

Fix: proxyReq.Header.Del("Accept-Encoding") before the upstream round-trip. The CDN should store one uncompressed representation per URL; Caddy handles per-client content encoding on egress.

Bug 2: matchVary was looking in the wrong place (pkg/httpcache/serve.go, rw.go)

When the response included Vary: Accept-Encoding, matchVary looked for Accept-Encoding in the response headers map — but Accept-Encoding is a request header and was never there. The cachedValue == "" branch silently continued, causing every request to match regardless of what encoding it accepted.

Fix: ToCacheValue now accepts the *http.Request and snapshots the Vary-relevant request header values into a new CacheValue.VaryRequestHeaders map[string]string field. matchVary now compares the incoming request's headers against that snapshot instead of looking in response headers. Legacy entries with no
VaryRequestHeaders are treated as misses so they repopulate correctly.

fix(pgs): dupe headers

chore(pgs): configurable max items for lru

chore: lint

chore: disable flaky test

chore(httpcache): cache-status cleanup and random bug fixes

chore: normalize header keys

chore: disable imgproxy tests

We don't have them setup properly atm

refactor(storage): no need for metadata just need content type

chore: rm dead code

refactor(storage): put accepts ObjectInfo instead of FileEntry

This is a symmetric change so everything uses the same objects

feat(pgs): record metrics for compressed file types

fix(pgs): check handler fast bail when domain is empty

refactor(httpcache): ignore routes should use no-store

fix(httpcache): verified heuristically cachable status codes

refactor(pgs): use http.ServeContent

refactor: new image proxy object that uses httputil reverse proxy

fix(pgs): typo

chore(pgs): make our cache age match our lru cache ttl

Fix cache metrics and update deps

fix(pgs): cdn routing to correct place

fix(pgs): admin check logic

chore(pgs): log key eviction

fix(pgs): use URL.Path so we ignore query params

feat(httpcache): ignore routes cfg

fix(pgs): prom names and set headers instead of add

chore(pgs): HEAD can share same key as GET

fix(pgs): 304 send correct status and headers

fix(pgs): httpcache must return cached headers when returning 304

fix(pgs): gauge name si unit

fix(pgs): httpcache rw.Write needs to report the num of bytes

refactor(pgs): custom rfc9111 compliant cache

chore: scripts

docs(tui): per file max size 100mb

feat(pgs): 100mb max file size

fix: test

feat(pgs): increase max file size to 250mb

fix(pgs): force redirect from root

fix(pgs): normalize ext

docs: rm auto form

chore(storage): cleanup

refactor: move shared/storage to storage

refactor: merge pboj and shared/storage

Update soju build to support webpush changes and upgrade to go 1.26

Fix host resolution for caddy

chore(pgs): require pico+ for auto-forms

chore(feeds): update retry attempt max in ls cmd

feat(pgs): auto-forms

fix(feeds): parse urls properly with comments

chore(feeds): extend retry number to 100

10 is too small since we count every 10 minutes

feat(pgs): all projects named `bin` will have their files deleted automatically after 2 weeks

This replicates the 0x0.st functionality where you can uploading any file and it'll eventually be deleted.

chore(pgs): setting acl is pico+ only

feat(pgs): http-pass ACL

This feature enables users to set their project to "http-pass" which requires a password to access the site.

The user can set the acl via:

`ssh pgs.sh acl {project} --type http-pass --acl {pass}`

When accessing the project the user will have to enter the password to access the site.  When access has been granted we set a cookie in the user's browser.  That cookie is valid for 24 hours.

feat(pgs): project names prefixed with `private-` will have new acl type "private"

For security reasons once a project is named `private-` it can never have its ACL type changed.

Right now we do not support making other projects private.

fix(pgs): ability to pipe files to pgs

docs(pgs): more usage for help page

chore(bouncer): bump soju to latest

fix(tui): revert 'q' to go back

This is conflicting with our text input boxes so I'm reverting for now

Chore: Unified Error Logs throughout the repo (#210)

fix(pssh): race in tests

fix(pssh): fix race

fix(pssh): tests

fix(pssh): normalize line endings for pty enabled ssh output

refactor(pgs): use dynamic port for ssh tests

feat(tui): 'q' goes to prev page

fix(pssh): normalize line ending with pty

fix(pssh): redundant sesh passing

fix(pssh): strip quotes after splitting on them

fix: properly parse ssh command args with quotes

chore: added test for pssh cmd parsing

chore(pssh): add error/print helper cmds

fix(pgs): downgrade storages to v0.0.16

http cache is being truncated on large files

Reference: https://github.com/darkweak/storages/issues/41

chore(pgs): test failure

chore(pgs): tests to prevent infinite redirects

Fix deps

Upgrade to go 1.25

feat(pubsub): round robin

refactor: move picosh/pubsub into repo

chore: more refactor cleanup

refactor: fix issues with imports and create shared/router pkg

refactor: migrate picosh/utils

feat(pipe): ?prefix= query param for pub and pipe

feat(auth): `/pubkeys/{user}` public endpoint

fix(pgs): inverted logic check for pico+

chore: gitignore

feat(pico): access_logs cli command

chore(tui): tokens text for Apple Terminal

chore(bouncer): motd updates

fix: identity is always pubkey so check it as well

fix(ssh): only use pubkey name if not ssh cert

chore(pipe): use ssh cert identity if available

chore(pipemgr): use cert location

feat(ssh): find pubkey name and use as identity for access logs

fix(tui.access_logs): start at last access with cmds to go to top or bot list

refactor: remove minio

We haven't used minio in months and if we did choose another object storage it wouldn't minio

docs(pgs): simplify pgs marketing

refactor(pgs): cache-control max-age and revalidate

We want to prevent the browser from caching stale static pages by requiring it to revalidate using ETag after 1 minute.

refactor(analytics): add indexes to improve query performance

fix(tui): border boundaries

style(prose): margin on the tags section

feat(tui.logs): allow logs to be manually scrolled

fix(tui.logs): filtered logs corruption

feat(prose): added horizontal rule to list parser

refactor(prose): replace ul/li with custom list impl

chore(prose,pastes): rotate caddy logs

fix(prose): list parser key is `draft`

chore(prose): list parser tests

chore(prose): add raw .lxt route

fix(prose): remove log line

feat(prose): lists are back bby

feat(pipe): web based rss feed

feat(pipe): monitors

This change introduces pipe monitors: a way to receive status updates on
your pipes.

chore(feeds): add deprecation notice for free pico users

fix(auth): updated_at required for pico+ feed posts

docs(readme): feeds txt

chore(prose): we no longer do anything with "prose" pgs project

This is from when we had a single object store.  We now have an object
store per service so creating a prose project is no longer required.