make it generic. clarify and reword the offset computation description.

Co-authored-by: arthurfranca <arthur.a.franca@gmail.com>

29.md

@@ -120,21 +120,21 @@ Clients can send these events to a relay in order to accomplish a moderation act
 
 Each moderation action uses a different kind and requires different arguments, which are given as tags. These are defined in the following table:
 
-| kind | name                | tags                                           |
-| ---  | ---                 | ---                                            |
-| 9000 | `put-user`          | `p` with pubkey hex and optional roles         |
-| 9001 | `remove-user`       | `p` with pubkey hex                            |
-| 9002 | `edit-metadata`     | fields from `kind:39000` to be modified        |
-| 9005 | `delete-event`      | `e` with event id hex                          |
-| 9007 | `create-group`      |                                                |
-| 9008 | `delete-group`      |                                                |
-| 9009 | `create-invite`     |                                                |
+| kind | name            | tags                                                                             |
+| ---  | ---             | ---                                                                              |
+| 9000 | `put-user`      | `p` with pubkey hex and optional roles                                           |
+| 9001 | `remove-user`   | `p` with pubkey hex                                                              |
+| 9002 | `edit-metadata` | fields to be modified, and optionally `unrestricted`, `open`, `visible` `public` |
+| 9005 | `delete-event`  | `e` with event id hex                                                            |
+| 9007 | `create-group`  |                                                                                  |
+| 9008 | `delete-group`  |                                                                                  |
+| 9009 | `create-invite` | arbitrary `code`                                                                 |
 
 It's expected that the group state (of who is an allowed member or not, who is an admin and with which permission or not, what are the group name and picture etc) can be fully reconstructed from the canonical sequence of these events.
 
 ### Group metadata events
 
-These events contain the group id in a `d` tag instead of the `h` tag. They MUST be created by the relay master key only and a single instance of each (or none) should exist at all times for each group. They are merely informative but should reflect the latest group state (as it was changed by moderation events over time).
+These events contain the group id in a `d` tag instead of the `h` tag. They MUST be created by the relay master key only (as stated by the [NIP-11](11.md) `"self"` pubkey) and a single instance of each (or none) should exist at all times for each group. They are merely informative but should reflect the latest group state (as it was changed by moderation events over time).
 
 - *group metadata* (`kind:39000`) (optional)
 

45.md

@@ -29,29 +29,119 @@ In case a relay uses probabilistic counts, it MAY indicate it in the response wi
 
 Whenever the relay decides to refuse to fulfill the `COUNT` request, it MUST return a `CLOSED` message.
 
+## HyperLogLog
+
+Relays may return an HyperLogLog value together with the count, hex-encoded.
+
+```
+["COUNT", <subscription_id>, {"count": <integer>, "hll": "<hex>"}]
+```
+
+This is so it enables merging results from multiple relays and yielding a reasonable estimate of reaction counts, comment counts and follower counts, while saving many millions of bytes of bandwidth for everybody.
+
+### Algorithm
+
+This section describes the steps a relay should take in order to return HLL values to clients.
+
+1. Upon receiving a filter, if it is eligible (see below) for HyperLogLog, compute the deterministic `offset` for that filter (see below);
+2. Initialize 256 registers to `0` for the HLL value;
+3. For all the events that are to be counted according to the filter, do this:
+    1. Read the byte at position `offset` of the event `pubkey`, its value will be the register index `ri`;
+    2. Count the number of leading zero bits starting at position `offset+1` of the event `pubkey` and add `1`;
+    3. Compare that with the value stored at register `ri`, if the new number is bigger, store it.
+
+That is all that has to be done on the relay side, and therefore the only part needed for interoperability.
+
+On the client side, these HLL values received from different relays can be merged (by simply going through all the registers in HLL values from each relay and picking the highest value for each register, regardless of the relay).
+
+And finally the absolute count can be estimated by running some methods I don't dare to describe here in English, it's better to check some implementation source code (also, there can be different ways of performing the estimation, with different quirks applied on top of the raw registers).
+
+### `offset` computation
+
+The `offset` (used in the HLL computation above) is derived deterministically from the filter sent by the client to the relay. The formula for obtaining the `offset` value is as follows:
+
+  1. Take the first tag attribute in the filter (with the `#` prefix);
+  2. From that, take its first item (it will be a string);
+  3. Obtain a 32-byte hex string from it:
+    - if the string is an event id or pubkey hex, use it as it is;
+    - if the string is an address (`<kind>:<pubkey>:<d-tag>`), use the `<pubkey>` part;
+    - if the string is anything else, hash it with a `sha256()` and take the result as a hex string;
+  4. From the 64-character hex string obtained before, take the character at position `32`;
+  5. Read that character as a base-16 number;
+  6. Add 8 to it: the result is the `offset`.
+
+For cases not covered above (filters without a tag attribute, for example), behavior isn't yet defined. This NIP may be modified later to specify those, but for now there isn't a use case that justifies using HLL in those circumstances.
+
+### Rationale
+
+The value of `offset` must be deterministic because that's the only way to allow relays to cache the HLL values so they don't have to count thousands of events from the database on every query. It also allows relays to precompute HLL values for any given target `<id>` or `<pubkey>` without having to store the events themselves directly, which can be handy in case of reactions, for example.
+
+### Common filters
+
+Some relays may decide to cache or precompute HLL values for some common canonical queries, and also to refrain from counting events that do not match these specs. These are such queries (this NIP can be modified later if more common useful queries are discovered and start being used):
+
+- **reaction count**: `{"#e": ["<id>"], "kinds": [7]}`
+- **repost count**: `{"#e": ["<id>"], "kinds": [6]}`
+- **quote count**: `{"#q": ["<id>"], "kinds": [1, 1111]}`
+- **reply count**: `{"#e": ["<id>"], "kinds": [1]}`
+- **comment count**: `{"#E": ["<id>"], "kinds": [1111]}`
+- **follower count**: `{"#p": ["<pubkey>"], "kinds": [3]}`
+
+Notice that these queries only include 1 tag attribute with always a single item in it, which means that implementors don't have to check the order in which these attributes show up in the filter.
+
+### Attack vectors
+
+One could mine a pubkey with a certain number of zero bits in the exact place where the HLL algorithm described above would look for them in order to artificially make its reaction or follow "count more" than others. For this to work a different pubkey would have to be created for each different target (event id, followed profile etc). This approach is not very different than creating tons of new pubkeys and using them all to send likes or follow someone in order to inflate their number of followers. The solution is the same in both cases: clients should not fetch these reaction counts from open relays that accept everything, they should base their counts on relays that perform some form of filtering that makes it more likely that only real humans are able to publish there and not bots or artificially-generated pubkeys.
+
+### `hll` encoding
+
+The value `hll` value must be the concatenation of the 256 registers, each being a uint8 value (i.e. a byte). Therefore `hll` will be a 512-character hex string.
+
+### Client-side usage
+
+This algorithm also allows clients to combine HLL responses received from relays with HLL counts computed locally from raw events. It's recommended that clients keep track of HLL values locally and add to these on each message received from relays. For example:
+
+  - a client wants to keep track of the number of reactions an event Z has received over time;
+  - the client has decided it will read reactions from relays A, B and C (the NIP-65 "read" relays of Z's author);
+  - of these, only B and C support HLL responses, so the client fetches both and merges them locally;
+  - then the client fetches all reaction events from A then manually applies each event to the HLL from the previous step, using the same algorithm described above;
+  - finally, the client reads the estimate count from the HLL and displays that to the user;
+  - optionally the client may store that HLL value (together with some "last-read-date" for relay A) and repeat the process again later:
+    - this time it only needs to fetch the new reactions from A and add those to the HLL
+    - and redownload the HLL values from B and C and just reapply them to the local value.
+
+This procedure allows the client to download much less data.
+
 ## Examples
 
-### Followers count
-
-```
-["COUNT", <query_id>, {"kinds": [3], "#p": [<pubkey>]}]
-["COUNT", <query_id>, {"count": 238}]
-```
-
-### Count posts and reactions
+### Count notes and reactions
 
 ```
 ["COUNT", <query_id>, {"kinds": [1, 7], "authors": [<pubkey>]}]
 ["COUNT", <query_id>, {"count": 5}]
 ```
 
-### Count posts approximately
+### Count notes approximately
 
 ```
 ["COUNT", <query_id>, {"kinds": [1]}]
 ["COUNT", <query_id>, {"count": 93412452, "approximate": true}]
 ```
 
+### Followers count with HyperLogLog
+
+```
+["COUNT", <subscription_id>, {"kinds": [3], "#p": [<pubkey>]}]
+["COUNT", <subscription_id>, {"count": 16578, "hll": "0607070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]
+```
+
+### Reaction counts with HyperLogLog
+
+```
+["COUNT", <subscription_id>, {"kinds": [7], "#e": [<id>]}]
+["COUNT", <subscription_id>, {"count": 2044, "hll": "01ef070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]
+```
+
 ### Relay refuses to count
 
 ```

46.md

@@ -97,18 +97,25 @@ Each of the following are methods that the _client_ sends to the _remote-signer_
 
 | Command                  | Params                                                                        | Result                                                                 |
 | ------------------------ | -------------------------------------------------                             | ---------------------------------------------------------------------- |
-| `connect`                | `[<remote-signer-pubkey>, <optional_secret>, <optional_requested_permissions>]`        | "ack" OR `<required-secret-value>`                                                                  |
+| `connect`                | `[<remote-signer-pubkey>, <optional_secret>, <optional_requested_perms>]`     | `"ack"` OR `<required-secret-value>`                                   |
 | `sign_event`             | `[<{kind, content, tags, created_at}>]`                                       | `json_stringified(<signed_event>)`                                     |
-| `ping`                   | `[]`                                                                          | "pong"                                                                 |
-| `get_public_key`         | `[]`                                                                          | `<user-pubkey>`                                                         |
+| `ping`                   | `[]`                                                                          | `"pong"`                                                               |
+| `get_public_key`         | `[]`                                                                          | `<user-pubkey>`                                                        |
 | `nip04_encrypt`          | `[<third_party_pubkey>, <plaintext_to_encrypt>]`                              | `<nip04_ciphertext>`                                                   |
 | `nip04_decrypt`          | `[<third_party_pubkey>, <nip04_ciphertext_to_decrypt>]`                       | `<plaintext>`                                                          |
 | `nip44_encrypt`          | `[<third_party_pubkey>, <plaintext_to_encrypt>]`                              | `<nip44_ciphertext>`                                                   |
 | `nip44_decrypt`          | `[<third_party_pubkey>, <nip44_ciphertext_to_decrypt>]`                       | `<plaintext>`                                                          |
+| `switch_relays`          | `[]`                                                                          | `["<relay-url>", "<relay-url>", ...]` OR `null`                        |
 
 ### Requested permissions
 
-The `connect` method may be provided with `optional_requested_permissions` for user convenience. The permissions are a comma-separated list of `method[:params]`, i.e. `nip44_encrypt,sign_event:4` meaning permissions to call `nip44_encrypt` and to call `sign_event` with `kind:4`. Optional parameter for `sign_event` is the kind number, parameters for other methods are to be defined later. Same permission format may be used for `perms` field of `metadata` in `nostrconnect://` string.
+The `connect` method may be provided with `optional_requested_perms` for user convenience. The permissions are a comma-separated list of `method[:params]`, i.e. `nip44_encrypt,sign_event:4` meaning permissions to call `nip44_encrypt` and to call `sign_event` with `kind:4`. Optional parameter for `sign_event` is the kind number, parameters for other methods are to be defined later. Same permission format may be used for `perms` field of `metadata` in `nostrconnect://` string.
+
+### Switching relays
+
+At all times, the _remote-signer_ should be in control of what relays are being used for the connection between it and the _client_. Therefore it should be possible for it to evolve its set of relays over time as old relays go out of operation and new ones appear. Even more importantly, in the case of the connection initiated by the _client_ the client may pick relays completely foreign to the _remote-signer_'s preferences, so it must be possible for it to switch those immediately.
+
+Therefore, compliant clients should send a `switch_relays` request immediately upon establishing a connection (always, or at reasonable intervals). Upon receiving such requests, the _remote-signer_ should reply with its updated list of relays, or `null` if there is nothing to be changed. Immediately upon receiving an updated relay list, the _client_ should update its local state and send further requests on the new relays. The `remote-signer` should then be free to disconnect from the previous relays if that is desired.
 
 ## Response Events `kind:24133`
 

73.md

@@ -18,6 +18,7 @@ There are certain established global content identifiers such as [Book ISBNs](ht
 | URLs                   | "`<URL, normalized, no fragment>`"                         | "web"                    |
 | Books                  | "isbn:`<id, without hyphens>`"                             | "isbn"                   |
 | Geohashes              | "geo:`<geohash, lowercase>`"                               | "geo"                    |
+| Countries              | "iso3166:`<code, uppercase>`"                              | "iso3166"                |
 | Movies                 | "isan:`<id, without version part>`"                        | "isan"                   |
 | Papers                 | "doi:`<id, lowercase>`"                                    | "doi"                    |
 | Hashtags               | "#`<topic, lowercase>`"                                    | "#"                      |
@@ -43,6 +44,21 @@ For the webpage "https://myblog.example.com/post/2012-03-27/hello-world" the "i"
 ]
 ```
 
+### Geohashes:
+
+- Geohash: `["i", "geo:ezs42e44yx96"]` - https://www.movable-type.co.uk/scripts/geohash.html
+
+Geohashes are a geocoding system that encodes geographic locations into short strings of letters and digits. They MUST be lowercase.
+
+### Countries:
+
+ISO 3166 codes can reference countries (ISO 3166-1 alpha-2) or subdivisions like states/provinces (ISO 3166-2).
+
+- Country (Venezuela): `["i", "iso3166:VE"]`
+- Subdivision (California, USA): `["i", "iso3166:US-CA"]`
+
+ISO 3166 codes MUST be uppercase. More info: https://en.wikipedia.org/wiki/ISO_3166
+
 ### Books:
 
 - Book ISBN: `["i", "isbn:9780765382030"]` - https://isbnsearch.org/isbn/9780765382030

85.md

@@ -0,0 +1,132 @@
+NIP-85
+======
+
+Trusted Assertions
+------------------
+
+`draft` `optional`
+
+Certain Webs of Trust calculations require access to a large volume of events and/or computing power, making it virtually impossible to perform them directly on clients. This NIP allows users to offload such calculations to declared trusted service providers, and for these providers to publish signed "Trusted Assertion" events for the user client's consumption.
+
+## Assertion Events
+
+Trusted Assertions are always addressable (replaceable) events with the `d` tag pointing to the "subject" of the assertion. This NIP currently recognizes three distinct target "subjects" on which such calculations can be performed: *pubkeys*, *regular events*, and *addressable events*. Each subject type is mapped to an event kind: 
+
+| Subject            | Event Kind     | `d` tag value     |
+| ------------------ | -------------- | ----------------- |
+| User               | 30382          | `<pubkey>`        |
+| Event              | 30383          | `<event_id>`      |
+| Addressable Event  | 30384          | `<event_address>` |
+| NIP-73 Identifier  | 30385          | `<i-tag>`         | 
+
+Calculation results are saved in pre-defined tags whose syntax and semantics are agreed upon by providers and clients. 
+
+Example of ranking a pubkey with a web of trust score of `89`: 
+
+```jsonc
+{
+  "kind": 30382,
+  "tags": [
+    ["d", "e88a691e98d9987c964521dff60025f60700378a4879180dcbbb4a5027850411"], // target user's public key
+    ["rank", "89"],
+  ],
+  "content": "",
+  //...
+}
+```
+
+## Kind 30382: Users as Subject: 
+
+The following result types have been declared:
+
+| Result type             | Tag name               | Tag value format  |
+| ----------------------- | ---------------------- | ----------------- |
+| Follower Count          | `followers`            | int               |
+| User Rank               | `rank`                 | int, norm 0-100   |
+| First Post Time         | `first_created_at`     | unix timestamp    | 
+| Post Count              | `post_cnt`             | int               | 
+| Reply Count             | `reply_cnt`            | int               | 
+| Reactions Count         | `reactions_cnt`        | int               |
+| Zap Amount Received     | `zap_amt_recd`         | int, sats         |
+| Zap Amount Sent         | `zap_amt_sent`         | int, sats         | 
+| Zap Number Received     | `zap_cnt_recd`         | int               |
+| Zap Number Sent         | `zap_cnt_sent`         | int               |
+| Avg Zap Amount/day recd | `zap_avg_amt_day_recd` | int, sats         |
+| Avg Zap Amount/day sent | `zap_avg_amt_day_sent` | int, sats         |
+| Reports Received        | `reports_cnt_recd`     | int               |
+| Reports Sent            | `reports_cnt_sent`     | int               |
+| Common Topics           | `t`                    | string            |
+| Generally active start  | `active_hours_start`   | int, 0-24, UTC    |
+| Generally active end    | `active_hours_end`     | int, 0-24, UTC    |
+
+Each provider can offer their own ways to calculate such values. For instance, the Follower Count of one trust provider might remove the user's muted public keys while another provider keeps them. Users can then choose how they want to see this information in their preferred client by picking a provider that aligns with their view. 
+
+## Kind 30383: Events as Subject
+
+Providers can rate individual events with the following tags:  
+
+| Result type             | Tag name               | Tag value format  |
+| ----------------------- | ---------------------- | ----------------- |
+| Event Rank              | `rank`                 | int, norm 0-100   | 
+| Event Comment Count     | `comment_cnt`          | int               |
+| Event Quote Count       | `quote_cnt`            | int               |
+| Event Repost Count      | `repost_cnt`           | int               |
+| Event Reaction Count    | `reaction_cnt`         | int               |
+| Event Zap Count         | `zap_cnt`              | int               |     
+| Event Zap Amount        | `zap_amount`           | int, sats         |
+
+## Kind 30384: Addressables as Subject
+
+Providers can rate all versions of addressable events using the following tags: 
+
+| Result type               | Tag name               | Tag value format  |
+| ------------------------- | ---------------------- | ----------------- |
+| Address Rank              | `rank`                 | int, norm 0-100   | 
+| Address Comment Count     | `comment_cnt`          | int               |
+| Address Quote Count       | `quote_cnt`            | int               |
+| Address Repost Count      | `repost_cnt`           | int               |
+| Address Reaction Count    | `reaction_cnt`         | int               |
+| Address Zap Count         | `zap_cnt`              | int               |     
+| Address Zap Amount        | `zap_amount`           | int, sats         |
+
+## Kind 30385: External identifier as Subject
+
+Providers can rate books, locations, movies, websites, and hashtags using [NIP-73](73.md) identifiers. 
+
+| Result type       | Tag name               | Tag value format  |
+| ----------------- | ---------------------- | ----------------- |
+| Rank              | `rank`                 | int, norm 0-100   | 
+| Comment Count     | `comment_cnt`          | int               |
+| Reaction Count    | `reaction_cnt`         | int               |
+
+NIP-73 `k` tags should be added to the event as well.
+
+## Declaring Trusted Service Providers
+
+Kind `10040` lists the user's authorized providers for each result. Each `kind:tag` is followed by the `pubkey` of the service and the relay where the results are published. Users can specify these publicly or privately by JSON-stringifying and encrypting the tag list in the `.content` using NIP-44. 
+
+```js
+{
+  "kind": 10040,
+  "tags": [
+    ["30382:rank", "4fd5e210530e4f6b2cb083795834bfe5108324f1ed9f00ab73b9e8fcfe5f12fe", "wss://nip85.nostr.band"],
+    ["30382:rank", "3d842afecd5e293f28b6627933704a3fb8ce153aa91d790ab11f6a752d44a42d", "wss://nostr.wine"],
+    ["30382:zap_amt_sent", "4fd5e210530e4f6b2cb083795834bfe5108324f1ed9f00ab73b9e8fcfe5f12fe", "wss://nip85.nostr.band"],
+  ],
+  "content": nip44Encrypt(JSON.stringify([
+    ["30383:rank", "4fd5e210530e4f6b2cb083795834bfe5108324f1ed9f00ab73b9e8fcfe5f12fe", "wss://nip85.nostr.band"],
+    ["30384:rank", "4fd5e210530e4f6b2cb083795834bfe5108324f1ed9f00ab73b9e8fcfe5f12fe", "wss://nip85.nostr.band"],
+  ]),
+  //...
+}
+```
+
+If the provider offers several algorithms or multiple points of view of an algorithm, the key listed in each tag SHOULD point to the key created for each algorithm or point of view. 
+
+## Final Considerations
+
+Service providers SHOULD update Trusted Assertions as fast as new information arrives, but only if the contents of each event actually change to avoid re-downloading the same information.
+
+Service providers MAY limit access to the results by using paid relays.
+
+In TAs, `p`, `e`, and `a` tags with the same value as the `d` tag MAY be used to add a relay hint to the home relay of that user or event.

README.md

@@ -92,6 +92,7 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 - [NIP-78: Application-specific data](78.md)
 - [NIP-7D: Threads](7D.md)
 - [NIP-84: Highlights](84.md)
+- [NIP-85: Trusted Assertions](85.md)
 - [NIP-86: Relay Management API](86.md)
 - [NIP-87: Ecash Mint Discoverability](87.md)
 - [NIP-88: Polls](88.md)
@@ -260,6 +261,9 @@ They exist to document what may be implemented by [Nostr](https://github.com/nos
 | `30312`       | Interactive Room                | [53](53.md)                            |
 | `30313`       | Conference Event                | [53](53.md)                            |
 | `30315`       | User Statuses                   | [38](38.md)                            |
+| `30382`       | User Trusted Assertion          | [85](85.md)                            |
+| `30383`       | Event Trusted Assertion         | [85](85.md)                            |
+| `30384`       | Addressable Trusted Assertion   | [85](85.md)                            |
 | `30388`       | Slide Set                       | [Corny Chat][cornychat-slideset]       |
 | `30402`       | Classified Listing              | [99](99.md)                            |
 | `30403`       | Draft Classified Listing        | [99](99.md)                            |