pinned.red

Bookmarking with the AT Protocol

Introduction

Is bookmarking with the AT Protocol feasible?

I'm a habitual bookmarker, but I never come back to what I bookmarked. One day, I figured I should publish my bookmarks online if I wasn't going to do anything else with them. Someone's bound to find something they haven't seen before, I thought, as I tabbed back to Hacker News.

But how would I publish my bookmarks? Social bookmarking platforms such as Pinboard and Pinterest were still doing fine, even if they're not as relevant today. Some tech-savvy folks had found another use case for Notion and other note-taking apps as a power-user-friendly bookmark manager. And some went further and made a website they could call their own.

I kept circling back on this idea as I went job searching, until I came across an article on Bluesky and Nostr. I had heard of Bluesky and its AT Protocol in passing, but this article, an overview of the trade-offs ATProto designers and Nostr designers made in creating their respective federation protocols, convinced me to look into Bluesky's work further.

The AT Protocol specification left me with a good impression of what the Bluesky team is building, so much so that I decided I would use ATProto to publish my bookmarks, and maybe get a bookmark manager out of it. I still struggle to explain my rationale for choosing ATProto, but here's what I've settled on for now: I chose to use ATProto because...

This website is my proof of concept for managing public bookmarks using the AT Protocol. I'm tentatively calling it pinned.red. pinned.red is comprised of:

I'd need to add a few more things, like a server backend, before claiming that I made an app. Nevertheless, this proof of concept does enough to demonstrate how I can publish my bookmarks online using ATProto, and how a social bookmarking platform could possibly work on ATProto.

A Primer on ATProto Records and Lexicon

User data on the AT Protocol, and how it can be remixed

By default, each Bluesky (app) user is given an SQLite database. When a Bluesky user makes a post, that post is written to that user's database, or repository in ATProto terms. What makes ATProto repositories special is that access to the repositories is not hidden behind a central API controlled by Bluesky. App developers don't need to pay Bluesky for access; by design users' repositories can be read and written to by any independent app (although apps cannot write data without user approval).

A user's repository stores records of the user's Bluesky posts, likes, follows, etc. But how would we tell which records are for Bluesky posts and which are for Bluesky likes? Well, let's take a look at an example record: 

{
  "text": "logging on for my shift at the posting factory",
  "$type": "app.bsky.feed.post",
  "langs": [
    "en"
  ],
  "createdAt": "2023-07-03T18:11:26.814Z"
}

a record with type "app.bsky.feed.post" (view in PDSls)

This record has been serialized into JSON. It has a "$type" field; for this record, it's set to "app.bsky.feed.post". Can we assume from reading its "$type" field that this record represents a Bluesky post? Not quite. This record identifies itself as a Bluesky post, but from the record alone, we can't tell whether it represents a valid Bluesky post. In order to check the record's validity, we'll need to refer to its Lexicon definition.

Lexicon is the AT Protocol's schema definition language. Lexicon is based on JSON and is mainly used to define how repository records should be structured. Schema definitions for records (and API methods which we'll discuss later) are contained in Lexicon files.

Let's look at the Lexicon definition for our record. From reading its "$type" field, we learned that the record should map to the app.bsky.feed.post Lexicon definition. Currently, this Lexicon definition, and the Lexicon file it's contained in, can be found in the bluesky‑social/atproto GitHub project. Below, I've copied over the Lexicon file, but omitted less pertinent fields: 

{
  "lexicon": 1,
  "id": "app.bsky.feed.post",
  "defs": {
    "main": {
      "type": "record",
      "description": "Record containing a Bluesky post.",
      "key": "tid",
      "record": {
        "type": "object",
        "required": ["text", "createdAt"],
        "properties": {
          "text": {
            "type": "string",
            "maxLength": 3000,
            "maxGraphemes": 300,
            "description": "The primary post content. May be an empty string, if there are embeds."
          },
          "langs": {
            "type": "array",
            "description": "Indicates human language of post primary text content.",
            "maxLength": 3,
            "items": {
              "type": "string",
              "format": "language"
            }
          },
          "createdAt": {
            "type": "string",
            "format": "datetime",
            "description": "Client-declared timestamp when this post was originally created."
          },
          ...
        }
      }
    },
    "replyRef": {
      ...
    },
    "entity": {
      ...
    },
    "textSlice": {
      ...
    }
  }
}

the Lexicon file with the app.bsky.feed.post Lexicon definition (view original in GitHub)

Here's what we're looking for to validate our record:

Well then, is our record valid according to the app.bsky.feed.post Lexicon definition? Yes it is, our record contains the required properties/fields and each of them conforms to their specified type and constraints.

Now we can tell which repository records are valid Bluesky posts and which are valid Bluesky likes. Or rather, we can have ATProto apps identify and validate records for us. For example, embed.bsky.app looks up a record by URI, and if it finds a valid, "app.bsky.feed.post" record, it parses the record as a Bluesky post and creates an HTML embed.

embed.bsky.app, an app that generates code to embed a Bluesky post in a website

So far, I've talked about repositories and records as if they're intrinsic to Bluesky only; I said that each Bluesky user has a repository, and each repository stores records of Bluesky posts, likes, and so forth.

But repositories can store more than just "Bluesky" records. Repositories can store records of drawings, longform articles, and other content outside of microblogging.

The developers of the PinkSea and WhiteWind apps authored their own Lexicon definitions for records representing artwork or blog posts. Perhaps you're now thinking that "PinkSea" and "WhiteWind" records are only intended to be used by PinkSea and WhiteWind, respectively, since the apps are, for lack of a better term, official. However, any app can become a "PinkSea" or "WhiteWind" app. After all, some of us use alternative Bluesky clients like deck.blue over the official Bluesky app. Just as some apps could recreate the Bluesky experience from its Lexicon definitions, other apps can recreate the PinkSea and WhiteWind experiences from their Lexicon definitions.

Keep in mind, however, that Lexicon definitions allow records to be interoperable with any ATProto app, in every form. While reusing records in alternative clients is one way developers can remix an ATProto app using its related records, developers can be more creative and extend both the original app's feature set and its records' schemas.

Incidentally, some developers of alternative Bluesky clients have already done just that; they implemented pinned posts before Bluesky by "extending" the app.bsky.actor.profile Lexicon definition with a "pinnedPost" field. Meanwhile, WhiteWind uses Bluesky post records to implement comment sections, and sill.social searches for trending links across Bluesky as part of its news aggregator service.

Hopefully this primer covered enough for you to make sense of ATProto records and Lexicon. I went over how each Bluesky user (and anyone with an ATProto account) has a repository that stores user data in the form of records, how Lexicon allows apps to validate users' records, and how by publishing Lexicon definitions for the records their apps interact with, developers enable interoperability between each other's apps and user records. New kinds of apps are being introduced to the growing ATProto ecosystem thanks to the interoperability granted by Lexicon definitions.

The red.pinned.concept Lexicon

Primitives for managing bookmarks

My proof of concept for bookmark management, pinned.red, has its own Lexicon definitions:

Lexicon definitions describe records and API methods; I'll group the pinned.red Lexicon definitions into those two categories:

In this section, I'll mainly go over the Lexicon definitions for pinned.red records. I'll revisit the Lexicon definitions for pinned.red API methods in upcoming sections.

According to pinned.red, a bookmark looks like this:

{
  "uri": "https://en.wikipedia.org/wiki/Bookmark_(digital)",
  "$type": "red.pinned.concept.bookmark",
  "title": "Bookmark (digital) - Wikipedia",
  "createdAt": "1970-01-01T00:00:00Z"
}

a record with type "red.pinned.concept.bookmark"

The Lexicon definition for that "bookmark" record, and other "red.pinned.concept.bookmark" records, looks like this: 

{
  "type": "record",
  "description": "Record declaring a web bookmark.",
  "key": "tid",
  "record": {
    "type": "object",
    "required": ["uri", "createdAt"],
    "properties": {
      "uri": {
        "type": "string",
        "description": "URI that the bookmark refers to.",
        "format": "uri"
      },
      "title": {
        "type": "string",
        "description": "Title of the bookmark."
      },
      "createdAt": {
        "type": "string",
        "description": "Creation timestamp.",
        "format": "datetime"
      }
    }
  }
}

the red.pinned.concept.bookmark Lexicon definition (view Lexicon file)

A pinned.red bookmark has a URI, a creation timestamp, and, optionally, a title. Other bookmarks by web browsers have those three properties and more, but for pinned.red's bookmark schema I preferred to keep it simple. It would be nice to have comments or tags in the future...

Each pinned.red bookmark is a member of a pinned.red collection. pinned.red collections are bookmark folders, basically. A "collection" record looks like this: 

{
  "head": "at://did:plc:ewvi7nxzyoun6zhxrhs64oiz/red.pinned.concept.collectionmember/3jzfcijpj2z2a",
  "$type": "red.pinned.concept.collection",
  "title": "Collection title",
  "createdAt": "1970-01-01T00:00:00Z"
}

a record with type "red.pinned.concept.collection"

A pinned.red collection can have a title, a creation timestamp, and... a "head"? How does a collection "store" its bookmarks then? Well, the collection's "head" is a representation of the collection's bookmarks, sort of.

I'll explain what the value of "head" means when we get to the next Lexicon definition after collections, but for now, think of a pinned.red collection as a folded list of bookmarks. The first bookmark is written above the topmost fold, but to read the next bookmark you have to unfold the list. Unfold again for each subsequent bookmark and eventually you'll read every bookmark in the collection.

Here's the Lexicon definition for a "red.pinned.concept.collection" record: 

{
  "type": "record",
  "description": "Record declaring a collection of bookmarks and (sub)collections. A collection is modeled as an ordered list of its members; copies of members are allowed.",
  "key": "tid",
  "record": {
    "type": "object",
    "required": ["createdAt"],
    "properties": {
      "head": {
        "type": "string",
        "description": "First member of the collection.",
        "format": "at-uri"
      },
      "title": {
        "type": "string",
        "description": "Title of the collection."
      },
      "createdAt": {
        "type": "string",
        "description": "Creation timestamp.",
        "format": "datetime"
      }
    }
  }
}

the red.pinned.concept.collection Lexicon definition (view Lexicon file)

Hmm, there's some details on pinned.red collections I neglected to mention:

As for why the red.pinned.concept.collection Lexicon definition has a "head" property instead of a "members" property? It's due to concerns with record size. I can't predict how many bookmarks a pinned.red user will add to a collection, but I can imagine they could add a thousand before they became aware. On my web browser, I don't bother changing the default bookmark location, and now I have thousands of bookmarks in one folder.

A "red.pinned.concept.collection" record would grow much larger if apps could write multiple bookmarks and (sub)collections to it, to the point where its size becomes a performance bottleneck. Guidelines on size constraints for records (and other ATProto primitives) are still evolving, but I'd rather take caution than accidently breach some technical limitation.

Let's return to the "head" property of a pinned.red collection. 

"head": {
  "type": "string",
  "description": "First member of the collection.",
  "format": "at-uri"
}

the "head" property as defined by red.pinned.concept.collection

Instead of an array of bookmarks and collections, the "head" property is a string — an AT‑URI string.

AT‑URIs (or at:// URIs) are a subset of URIs for identifying resources on the AT Protocol, like repository records. AT‑URIs are often used in records to reference other records, forming relationships between record types. For a "red.pinned.concept.collection" record, its "head" AT‑URI is a reference to a "red.pinned.concept.collectionmember" record.

I need a better analogy to illustrate how pinned.red collections "stores" its members... but what if I told you that a pinned.red collection is fundamentaly a linked list? That should clear things up, right?

...

Okay, once I explain what "red.pinned.concept.collectionmember"s are for, then this'll all make sense. Please bear with me.

A "red.pinned.concept.collectionmember" record does not represent a member of a collection, so to speak. If pinned.red bookmarks and collections are already viewed as collection members, then a "red.pinned.concept.collectionmember" record can be viewed as the relationship between a collection member and the collection it belongs to. A collection membership, one could say.

Structurally, a "red.pinned.concept.collectionmember" record is a node for a singly linked list. It can have a "next" field and a "subject" field, according to its Lexicon definition: 

{
  "type": "record",
  "description": "Record declaring that a bookmark or (sub)collection is included once in a collection's ordered list of members. Remaining list members can be found by using the `next` property. The record key of the collectionmember record must match the record key of the subject, and that record must be in the same repository.",
  "key": "tid",
  "record": {
    "type": "object",
    "required": ["subject", "next"],
    "properties": {
      "subject": {
        "type": "string",
        "description": "The actual member — a bookmark or a (sub)collection.",
        "format": "at-uri"
      },
      "next": {
        "type": "string",
        "description": "Next member in the enclosing collection, except if `isLast` is set to `true`; then `next` points to the collection itself.",
        "format": "at-uri"
      },
      "isLast": {
        "type": "boolean",
        "description": "Declaration that this collection member is the last one in the enclosing collection.",
        "default": false
      }
    }
  }
}

the red.pinned.concept.collectionmember Lexicon definition (view Lexicon file)

Underneath those Lexicon conventions is a basic implementation of a linked list node. "next" gives the (AT‑URI) address of a subsequent node, unless the current node is last in the list/collection, and "subject" represents the actual value of the current node (a bookmark or a (sub)collection).

Now we can piece together what the "head" of a pinned.red collection is for. Given that a collection's "head" is analogous to a linked list's head, the first node in the list, we (or developers) can start from the "head" to iterate over all of the collection's members, just as one would iterate over a linked list from its own head. It's how apps know what members belong to the collection, just from reading a reference to a single member.

Suppose someone is iterating over a collection and they encounter a subcollection; if they iterate over that collection as well, then the multiple linked lists they are iterating over begin to resemble a tree. If they iterate over every subcollection and every subcollection in those subcollections and so on, they'll encounter every bookmark and collection descended from the "starting" collection.

In order for a pinned.red app to find all of a user's bookmarks and collections, it needs to know where to begin its search; the user's "red.pinned.concept.rootcollection" record indicates the starting point.

Here's the Lexicon definition for a "red.pinned.concept.rootcollection" record: 

{
  "type": "record",
  "description": "Record declaring a reference to an actor's root collection.",
  "key": "literal:self",
  "record": {
    "type": "object",
    "required": ["subject"],
    "properties": {
      "subject": {
        "type": "string",
        "description": "The collection.",
        "format": "at-uri"
      }
    }
  }
}

the red.pinned.concept.rootcollection Lexicon definition (view Lexicon file)

There's only one property: an AT‑URI to a pinned.red collection. The "key" field is more noteworthy: being set to "literal:self" indicates that a user can have only one "red.pinned.concept.rootcollection" record in their repository.

I've gone over every Lexicon definition for the pinned.red records a hypothetical pinned.red user can create.

If a pinned.red app gets made, a user will obviously want to create bookmarks, but they'll also want to organize them. The file directory continues to be a proven feature for grouping items under one designation, hence collections in pinned.red. With collections alone, users can create more granular classifications by nesting collections and ranking bookmarks within collections.

Together a user's pinned.red records form a hierarchy of bookmarks, even though from the perspective of the user's repository there's no such hierarchy.

To put another way, this...

example-handle.bsky.social's user repository
├─ red.pinned.concept.rootcollection/self
├─ red.pinned.concept.collection/examplecollec
├─ red.pinned.concept.bookmark/examplebookma
├─ red.pinned.concept.collectionmember/examplebookma
├─ red.pinned.concept.collection/7estedcollect
├─ red.pinned.concept.collectionmember/7estedcollect
├─ red.pinned.concept.bookmark/7estedbookmar
├─ red.pinned.concept.collectionmember/7estedbookmar
├─ red.pinned.concept.bookmark/7estedbookma2
└─ red.pinned.concept.collectionmember/7estedbookma2

an example repository listing with several pinned.red records represented by their $types and record keys

...can be interpreted as this...

red.pinned.concept.collection/examplecollec
├─ red.pinned.concept.bookmark/examplebookma
└─ red.pinned.concept.collection/7estedcollect
   ├─ red.pinned.concept.bookmark/7estedbookmar
   └─ red.pinned.concept.bookmark/7estedbookma2

a hierarchy of pinned.red bookmarks and collections formed by records from the example repository

Next I'll go over how users and developers can create pinned.red records by making calls to pinned.red API methods.

Log In to pinned.red

Access your bookmarks

In addition to Lexicon definitions, I've made some interactive widgets; if you have an ATProto account, you can use the widgets on this website to create your own collections of pinned.red bookmarks.

But first, you'll need to give permission for this website to write pinned.red records on your behalf:

{{ isLoggedIn ? 'Already Logged In' : 'Log In' }} Log Out

When you submit your Bluesky handle, you'll be redirected to another form that asks for your Bluesky password to verify your identity. After that, you'll be asked to confirm that you authorize this website to write to your repository. If you do confirm, you'll be redirected back to this website and the login form will be disabled ⸺ you're "logged in" now.

Creating Bookmarks and Collections of Bookmarks

Without writing records "by hand"

An API isn't required to create a pinned.red bookmark, but it makes writing valid "bookmark" records more convenient.

Each pinned.red API method is "documented" by a Lexicon definition. The Lexicon definitions tell developers how calls to the API should be formatted (e.g. what parameters are required, what to include in the request body) and what responses the API can return (e.g. binary data, error messages).

We can create a pinned.red bookmark using the red.pinned.concept.createBookmark API method; let's take a look at its Lexicon definition: 

{
  "type": "procedure",
  "description": "Create a bookmark.",
  "input": {
    "encoding": "application/json",
    "schema": {
      "type": "object",
      "required": ["actor", "uri"],
      "properties": {
        "actor": {
          "type": "string",
          "description": "Handle or DID of the actor (aka, current account).",
          "format": "at-identifier"
        },
        "uri": {
          "type": "string",
          "description": "URI that the bookmark refers to.",
          "format": "uri"
        },
        "title": {
          "type": "string",
          "description": "Title of the bookmark."
        },
        "collection": {
          "type": "string",
          "description": "Existing collection that the new bookmark should belong to.",
          "format": "at-uri"
        },
        "index": {
          "type": "integer",
          "description": "The position among existing collection members where the new bookmark should be placed.",
          "minimum": 0
        }
      }
    }
  },
  "output": {
    "encoding": "application/json",
    "schema": {
      "type": "object",
      "properties": {
        "ok": { "type": "boolean", "default": true }
      }
    }
  },
  "errors": [{ "name": "no" }]
}

the red.pinned.concept.createBookmark Lexicon definition (view Lexicon file)

The "properties" field within "input" specifies what "arguments" should be passed to the "createBookmark" API method. Therefore, when we call the "createBookmark" API method, we should pass:

Normally, we would send our pinned.red API calls to HTTP endpoints (as ATProto apps expose their APIs via HTTP), but for this proof of concept, we're going to make API calls via a wrapper library, XRPCMockup.js. The wrapper library is as close to a compliant implementation of pinned.red API methods as it can be without creating HTTP endpoints.

So instead of making an HTTP request, we'll make a function call using the wrapper library to create a pinned.red bookmark. This form will call the createBookmark function from the wrapper library once it's submitted:

Create Bookmark

And this form will call the createCollection library function, allowing us to create a pinned.red collection:

Create Collection

You can read the Lexicon definition for the red.pinned.concept.createCollection API method here.

A Simple pinned.red Client

Not a reference implementation

Any pinned.red bookmarks you create will show up as records in your repository. You can view the records with a repository browser like PDSls, atproto-browser.vercel.app, and astrolabe.

If you'd rather use something closer to a traditional file browser, with bookmark files arranged within a hierarchy of collection folders, I've made a basic bookmark manager (please read the caveats before using it):

With the bookmark manager, you can navigate to different pinned.red collections, access websites you bookmarked, and delete bookmarks and collections.

You can also copy the AT‑URI of a bookmark's or collection's underlying record by clicking the associated link button () in the Actions column of the bookmark manager. This AT‑URI can be used in the forms from the previous section.