Yes. I don't know how well documented it is, but there is an API (well, system keyspace) that can configure the database to log transactions for a selected key range (up to and including the whole database) into another selected key range. It is used by backup and asynchronous replication tools. The format of the configuration keys and transaction logs should be considered less stable than the core key/value API, which basically never breaks backward compatibility, so by using it you are shouldering a maintenance burden to keep up with e.g. changes in the log format and new types of mutations when new versions of FoundationDB come along. So it shouldn't be used too casually. But it's there.
Alternatively, your application or layer can use the "versionstamp" atomic operations to write its own ordered log of what it is doing, or other indexing tricks. Depending on your data model this might be able to be much more efficient. For example, for external indexing you probably don't need to preserve a history of prior values but only be able to identify all the values that have changed. This can be done with a very simple and compact index that doesn't need to duplicate all the data to be indexed.
> Alternatively, your application or layer can use the "versionstamp" atomic operations to write its own ordered log of what it is doing, or other indexing tricks.
I'm not sure I understand.
Are you suggesting having a second key space at `ops/{VERSIONSTAMP}` or something where values contain enough information about the operation to be able to process changes in an indexer? The indexer could then clean up after itself, deleting the operations once they had been ingested? ... Effectively using a portion of the keyspace as a queue?
Yes. Or if it's not important for the indexer to process things chronologically, you could just have an index of the primary key (only) of records that haven't been indexed.
If you are trying to make your external index MVCC, then you will want to carry some version information too.
This kind of question might be better served by the new community forum you can get to from the website!
Alternatively, your application or layer can use the "versionstamp" atomic operations to write its own ordered log of what it is doing, or other indexing tricks. Depending on your data model this might be able to be much more efficient. For example, for external indexing you probably don't need to preserve a history of prior values but only be able to identify all the values that have changed. This can be done with a very simple and compact index that doesn't need to duplicate all the data to be indexed.