make it generic. clarify and reword the offset computation description.

05.md

@@ -6,11 +6,11 @@ Mapping Nostr keys to DNS-based internet identifiers
 
 `final` `optional`
 
-On events of kind `0` (`user metadata`) one can specify the key `"nip05"` with an [internet identifier](https://datatracker.ietf.org/doc/html/rfc5322#section-3.4.1) (an email-like address) as the value. Although there is a link to a very liberal "internet identifier" specification above, the `<local-part>` part MUST only use characters `a-z0-9-_.`.
+On events of kind `0` (`user metadata`) one can specify the key `"nip05"` with an [internet identifier](https://datatracker.ietf.org/doc/html/rfc5322#section-3.4.1) (an email-like address) as the value. Although there is a link to a very liberal "internet identifier" specification above, NIP-05 assumes the `<local-part>` part will be restricted to the characters `a-z0-9-_.`, case-insensitive.
 
 Upon seeing that, the client splits the identifier into `<local-part>` and `<domain>` and use these values to make a GET request to `https://<domain>/.well-known/nostr.json?name=<local-part>`.
 
-The result should be a JSON document object with a key `"names"` that should then be a mapping of names to hex formatted public keys, in lowercase. If the public key for the given `<name>` matches the `pubkey` from the `user metadata` event, the client then concludes that the given pubkey can indeed be referenced by its identifier.
+The result should be a JSON document object with a key `"names"` that should then be a mapping of names to hex formatted public keys. If the public key for the given `<name>` matches the `pubkey` from the `user metadata` event, the client then concludes that the given pubkey can indeed be referenced by its identifier.
 
 ### Example
 
@@ -73,7 +73,7 @@ For example, if after finding that `bob@bob.com` has the public key `abc...def`,
 
 ### Public keys must be in hex format
 
-Keys must be returned in hex format, in lowercase. Keys in NIP-19 `npub` format are only meant to be used for display in client UIs, not in this NIP.
+Keys must be returned in hex format. Keys in NIP-19 `npub` format are only meant to be used for display in client UIs, not in this NIP.
 
 ### Showing just the domain as an identifier
 

22.md

@@ -192,7 +192,7 @@ A reply to a podcast comment:
     // this is a reference to the above comment
     ["e", "80c48d992a38f9c445b943a9c9f1010b396676013443765750431a9004bdac05", "wss://example.relay", "252f10c83610ebca1a059c0bae8255eba2f95be4d1d7bcfa89d7248a82d9f111"],
     // the parent comment kind
-    ["k", "1111"],
+    ["k", "1111"]
     ["p", "252f10c83610ebca1a059c0bae8255eba2f95be4d1d7bcfa89d7248a82d9f111"]
   ]
   // other fields

45.md

@@ -29,29 +29,119 @@ In case a relay uses probabilistic counts, it MAY indicate it in the response wi
 
 Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST return a `CLOSED` message.
 
+## HyperLogLog
+
+Relays may return an HyperLogLog value together with the count, hex-encoded.
+
+```
+["COUNT", <subscription_id>, {"count": <integer>, "hll": "<hex>"}]
+```
+
+This is so it enables merging results from multiple relays and yielding a reasonable estimate of reaction counts, comment counts and follower counts, while saving many millions of bytes of bandwidth for everybody.
+
+### Algorithm
+
+This section describes the steps a relay should take in order to return HLL values to clients.
+
+1. Upon receiving a filter, if it is eligible (see below) for HyperLogLog, compute the deterministic `offset` for that filter (see below);
+2. Initialize 256 registers to `0` for the HLL value;
+3. For all the events that are to be counted according to the filter, do this:
+    1. Read the byte at position `offset` of the event `pubkey`, its value will be the register index `ri`;
+    2. Count the number of leading zero bits starting at position `offset+1` of the event `pubkey` and add `1`;
+    3. Compare that with the value stored at register `ri`, if the new number is bigger, store it.
+
+That is all that has to be done on the relay side, and therefore the only part needed for interoperability.
+
+On the client side, these HLL values received from different relays can be merged (by simply going through all the registers in HLL values from each relay and picking the highest value for each register, regardless of the relay).
+
+And finally the absolute count can be estimated by running some methods I don't dare to describe here in English, it's better to check some implementation source code (also, there can be different ways of performing the estimation, with different quirks applied on top of the raw registers).
+
+### `offset` computation
+
+The `offset` (used in the HLL computation above) is derived deterministically from the filter sent by the client to the relay. The formula for obtaining the `offset` value is as follows:
+
+  1. Take the first tag attribute in the filter (with the `#` prefix);
+  2. From that, take its first item (it will be a string);
+  3. Obtain a 32-byte hex string from it:
+    - if the string is an event id or pubkey hex, use it as it is;
+    - if the string is an address (`<kind>:<pubkey>:<d-tag>`), use the `<pubkey>` part;
+    - if the string is anything else, hash it with a `sha256()` and take the result as a hex string;
+  4. From the 64-character hex string obtained before, take the character at position `32`;
+  5. Read that character as a base-16 number;
+  6. Add 8 to it: the result is the `offset`.
+
+For cases not covered above (filters without a tag attribute, for example), behavior isn't yet defined. This NIP may be modified later to specify those, but for now there isn't a use case that justifies using HLL in those circumstances.
+
+### Rationale
+
+The value of `offset` must be deterministic because that's the only way to allow relays to cache the HLL values so they don't have to count thousands of events from the database on every query. It also allows relays to precompute HLL values for any given target `<id>` or `<pubkey>` without having to store the events themselves directly, which can be handy in case of reactions, for example.
+
+### Common filters
+
+Some relays may decide to cache or precompute HLL values for some common canonical queries, and also to refrain from counting events that do not match these specs. These are such queries (this NIP can be modified later if more common useful queries are discovered and start being used):
+
+- **reaction count**: `{"#e": ["<id>"], "kinds": [7]}`
+- **repost count**: `{"#e": ["<id>"], "kinds": [6]}`
+- **quote count**: `{"#q": ["<id>"], "kinds": [1, 1111]}`
+- **reply count**: `{"#e": ["<id>"], "kinds": [1]}`
+- **comment count**: `{"#E": ["<id>"], "kinds": [1111]}`
+- **follower count**: `{"#p": ["<pubkey>"], "kinds": [3]}`
+
+Notice that these queries only include 1 tag attribute with always a single item in it, which means that implementors don't have to check the order in which these attributes show up in the filter.
+
+### Attack vectors
+
+One could mine a pubkey with a certain number of zero bits in the exact place where the HLL algorithm described above would look for them in order to artificially make its reaction or follow "count more" than others. For this to work a different pubkey would have to be created for each different target (event id, followed profile etc). This approach is not very different than creating tons of new pubkeys and using them all to send likes or follow someone in order to inflate their number of followers. The solution is the same in both cases: clients should not fetch these reaction counts from open relays that accept everything, they should base their counts on relays that perform some form of filtering that makes it more likely that only real humans are able to publish there and not bots or artificially-generated pubkeys.
+
+### `hll` encoding
+
+The value `hll` value must be the concatenation of the 256 registers, each being a uint8 value (i.e. a byte). Therefore `hll` will be a 512-character hex string.
+
+### Client-side usage
+
+This algorithm also allows clients to combine HLL responses received from relays with HLL counts computed locally from raw events. It's recommended that clients keep track of HLL values locally and add to these on each message received from relays. For example:
+
+  - a client wants to keep track of the number of reactions an event Z has received over time;
+  - the client has decided it will read reactions from relays A, B and C (the NIP-65 "read" relays of Z's author);
+  - of these, only B and C support HLL responses, so the client fetches both and merges them locally;
+  - then the client fetches all reaction events from A then manually applies each event to the HLL from the previous step, using the same algorithm described above;
+  - finally, the client reads the estimate count from the HLL and displays that to the user;
+  - optionally the client may store that HLL value (together with some "last-read-date" for relay A) and repeat the process again later:
+    - this time it only needs to fetch the new reactions from A and add those to the HLL
+    - and redownload the HLL values from B and C and just reapply them to the local value.
+
+This procedure allows the client to download much less data.
+
 ## Examples
 
-### Followers count
-
-```
-["COUNT", <query_id>, {"kinds": [3], "#p": [<pubkey>]}]
-["COUNT", <query_id>, {"count": 238}]
-```
-
-### Count posts and reactions
+### Count notes and reactions
 
 ```
 ["COUNT", <query_id>, {"kinds": [1, 7], "authors": [<pubkey>]}]
 ["COUNT", <query_id>, {"count": 5}]
 ```
 
-### Count posts approximately
+### Count notes approximately
 
 ```
 ["COUNT", <query_id>, {"kinds": [1]}]
 ["COUNT", <query_id>, {"count": 93412452, "approximate": true}]
 ```
 
+### Followers count with HyperLogLog
+
+```
+["COUNT", <subscription_id>, {"kinds": [3], "#p": [<pubkey>]}]
+["COUNT", <subscription_id>, {"count": 16578, "hll": "0607070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]
+```
+
+### Reaction counts with HyperLogLog
+
+```
+["COUNT", <subscription_id>, {"kinds": [7], "#e": [<id>]}]
+["COUNT", <subscription_id>, {"count": 2044, "hll": "01ef070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]
+```
+
 ### Relay refuses to count
 
 ```

47.md

@@ -371,7 +371,7 @@ Response:
     "result_type": "lookup_invoice",
     "result": {
         "type": "incoming", // "incoming" for invoices, "outgoing" for payments
-        "state": "pending", // can be "pending", "settled", "accepted" (for hold invoices), "expired" (for invoices) or "failed" (for payments), optional
+        "state": "pending", // can be "pending", "settled", "expired" (for invoices) or "failed" (for payments), optional
         "invoice": "string", // encoded invoice, optional
         "description": "string", // invoice's description, optional
         "description_hash": "string", // invoice's description hash, optional
@@ -420,7 +420,7 @@ Response:
         "transactions": [
             {
                "type": "incoming", // "incoming" for invoices, "outgoing" for payments
-               "state": "pending", // can be "pending", "settled", "accepted" (for hold invoices), "expired" (for invoices) or "failed" (for payments), optional
+               "state": "pending", // can be "pending", "settled", "expired" (for invoices) or "failed" (for payments), optional
                "invoice": "string", // encoded invoice, optional
                "description": "string", // invoice's description, optional
                "description_hash": "string", // invoice's description hash, optional
@@ -485,89 +485,6 @@ Response:
 }
 ```
 
-### `make_hold_invoice`
-
-Creates a hold invoice using a pre-generated preimage.
-
-Request:
-```jsonc
-{
-    "method": "make_hold_invoice",
-    "params": {
-        "amount": 123, // value in msats
-        "description": "string", // invoice's description, optional
-        "description_hash": "string", // invoice's description hash, optional
-        "expiry": 213 // expiry in seconds from time invoice is created for a payment to be initiated, optional. This does not determine how long a payment can be held (see `settle_deadline`)
-        "payment_hash": "string" // Payment hash for the payment generated from the preimage
-        "min_cltv_expiry_delta": 144 // The minimum CLTV delta to use for the final hop, optional
-    }
-}
-```
-
-Response:
-```jsonc
-{
-    "result_type": "make_hold_invoice",
-    "result": {
-        "type": "incoming", // "incoming" for invoices, "outgoing" for payments
-        "invoice": "string", // encoded invoice, optional
-        "description": "string", // invoice's description, optional
-        "description_hash": "string", // invoice's description hash, optional
-        "payment_hash": "string", // Payment hash for the payment
-        "amount": 123, // value in msats
-        "created_at": unixtimestamp, // invoice/payment creation time
-        "expires_at": unixtimestamp, // invoice expiration time, optional if not applicable
-        "metadata": {} // generic metadata that can be used to add things like zap/boostagram details for a payer name/comment/etc.
-    }
-}
-```
-
-### `cancel_hold_invoice`
-
-Cancels a hold invoice using the payment hash
-
-Request:
-```jsonc
-{
-    "method": "cancel_hold_invoice",
-    "params": {
-        "payment_hash": "string" // Payment hash for the payment generated from the preimage
-    }
-}
-```
-
-Response:
-```jsonc
-{
-    "result_type": "cancel_hold_invoice",
-    "result": {}
-}
-```
-
-### `settle_hold_invoice`
-
-Settles a hold invoice using the preimage
-
-
-Request:
-```jsonc
-{
-    "method": "settle_hold_invoice",
-    "params": {
-        "preimage": "string" // preimage for the payment
-    }
-}
-```
-
-Response:
-```jsonc
-{
-    "result_type": "settle_hold_invoice",
-    "result": {}
-}
-```
-
-
 ## Notifications
 
 ### `payment_received`
@@ -622,30 +539,6 @@ Notification:
 }
 ```
 
-### `hold_invoice_accepted`
-
-Description: Sent when a payer accepts (locks in) a hold invoice. To avoid locking up funds in channels the hold invoice SHOULD be settled or canceled within a few minutes of receiving this event.
-
-Notification:
-```jsonc
-{
-    "notification_type": "hold_invoice_accepted",
-    "notification": {
-        "type": "incoming",
-        "state": "accepted",  // optional
-        "invoice": "string", // encoded invoice
-        "description": "string", // invoice's description, optional
-        "description_hash": "string", // invoice's description hash, optional
-        "payment_hash": "string", // Payment hash for the payment
-        "amount": 123, // value in msats
-        "created_at": unixtimestamp, // invoice/payment creation time
-        "expires_at": unixtimestamp, // invoice expiration time
-        "settle_deadline": blocknumber, // invoice can only be safely settled or canceled before this block number.
-        "metadata": {} // generic metadata that can be used to add things like zap/boostagram details for a payer name/comment/etc.
-    }
-}
-```
-
 ## Example pay invoice flow
 
 0. The user scans the QR code generated by the **wallet service** with their **client** application, they follow a `nostr+walletconnect://` deeplink or configure the connection details manually.
@@ -775,16 +668,6 @@ Here are some properties that are recognized by some NWC clients:
 }
 ```
 
-### Example Hold Invoice Support Flow
-
-1. Client generates a 32-byte hex-encoded preimage.
-2. Computes SHA-256 to derive payment hash.
-3. Sends `make_hold_invoice` with payment hash and desired parameters.
-4. Waits for `hold_invoice_accepted` notification.
-5. Upon receiving notification, either:
-
-   * Calls `settle_hold_invoice` with the original preimage to release funds, or
-   * Calls `cancel_hold_invoice` with payment hash to abort.
 ### Deep-links
 
 Wallet applications can register deeplinks in mobile systems to make it possible to create a linking UX that doesn't require the user scanning a QR code or pasting some code.

51.md

@@ -135,6 +135,7 @@ Some clients have used these lists in the past, but they should work on transiti
     ["e", "340e0326b340e0326b4941ed78ba340e0326b4941ed78ba340e0326b49ed78ba"], // PWA
     ["a", "32267:d6dc95542e18b8b7aec2f14610f55c335abebec76f3db9e58c254661d0593a0c:com.example.app"] // Reference to parent software application
   ],
+  "content": "Example App is a decentralized marketplace for apps",
   "sig": "a9a4e2192eede77e6c9d24ddfab95ba3ff7c03fbd07ad011fff245abea431fb4d3787c2d04aad001cb039cb8de91d83ce30e9a94f82ac3c5a2372aa1294a96bd"
 }
 ```