Module hb_message.erl ¶

This module acts an adapter between messages, as modeled in the AO-Core protocol, and their uderlying binary representations and formats.

Description¶

Unless you are implementing a new message serialization codec, you should not need to interact with this module directly. Instead, use the hb_ao interfaces to interact with all messages. The dev_message module implements a device interface for abstracting over the different message formats.

hb_message and the HyperBEAM caches can interact with multiple different types of message formats:

Richly typed AO-Core structured messages.
Arweave transations.
ANS-104 data items.
HTTP Signed Messages.
Flat Maps.

This module is responsible for converting between these formats. It does so by normalizing messages to a common format: Type Annotated Binary Messages (TABM). TABMs are deep Erlang maps with keys than only contain either other TABMs or binary values. By marshalling all messages into this format, they can easily be coerced into other output formats. For example, generating a HTTP Signed Message format output from an Arweave transaction. TABM is also a simple format from a computational perspective (only binary literals and O(1) access maps), such that operations upon them are efficient.

The structure of the conversions is as follows:

Arweave TX/ANS-104 ==> dev_codec_ans104:from/1 ==> TABM
HTTP Signed Message ==> dev_codec_httpsig_conv:from/1 ==> TABM
Flat Maps ==> dev_codec_flat:from/1 ==> TABM

TABM ==> dev_codec_structured:to/1 ==> AO-Core Message
AO-Core Message ==> dev_codec_structured:from/1 ==> TABM

TABM ==> dev_codec_ans104:to/1 ==> Arweave TX/ANS-104
TABM ==> dev_codec_httpsig_conv:to/1 ==> HTTP Signed Message
TABM ==> dev_codec_flat:to/1 ==> Flat Maps
...

Additionally, this module provides a number of utility functions for manipulating messages. For example, hb_message:sign/2 to sign a message of arbitrary type, or hb_message:format/1 to print an AO-Core/TABM message in a human-readable format.

The hb_cache module is responsible for storing and retrieving messages in the HyperBEAM stores configured on the node. Each store has its own storage backend, but each works with simple key-value pairs. Subsequently, the hb_cache module uses TABMs as the internal format for storing and retrieving messages.

Test vectors to ensure the functioning of this module and the codecs that interact with it are found in hb_message_test_vectors.erl.

Function Index¶

commit/2	Sign a message with the given wallet.
commit/3
commitment/2	Extract a commitment from a message given a `committer` ID, or a spec message to match against.
commitment/3
commitment_devices/2	Return the devices for which there are commitments on a message.
committed/3	Return the list of committed keys from a message.
conversion_spec_to_req/2*	Get a codec device and request params from the given conversion request.
convert/3	Convert a message from one format to another.
convert/4
default_tx_list/0	Get the ordered list of fields as AO-Core keys and default values of the tx record.
default_tx_message/0*	Get the normalized fields and default values of the tx record.
filter_default_keys/1	Remove keys from a map that have the default values found in the tx record.
find_target/3	Implements a standard pattern in which the target for an operation is found by looking for a `target` key in the request.
format/1	Format a message for printing, optionally taking an indentation level to start from.
format/2
from_tabm/4*
id/1	Return the ID of a message.
id/2
id/3
is_signed_key/3	Determine whether a specific key is part of a message's commitments.
match/2	Check if two maps match, including recursively checking nested maps.
match/3
match/4
matchable_keys/1*
minimize/1	Remove keys from the map that can be regenerated.
minimize/2*
normalize/2*	Return a map with only the keys that necessary, without those that can be regenerated.
print/1	Pretty-print a message.
print/2*
restore_priv/3*	Add the existing `priv` sub-map back to a converted message, honoring any existing `priv` sub-map that may already be present.
signers/2	Return all of the committers on a message that have 'normal', 256 bit, addresses.
to_tabm/3*
type/1	Return the type of an encoded message.
uncommitted/1	Return the unsigned version of a message in AO-Core format.
uncommitted/2
unsafe_match/5*
verify/1	wrapper function to verify a message.
verify/2
verify/3
with_commitments/3	Filter messages that do not match the 'spec' given.
with_only_committed/2	Return a message with only the committed keys.
with_only_committers/2	Return the message with only the specified committers attached.
with_only_committers/3
without_commitments/3	Filter messages that match the 'spec' given.

Function Details¶

commit/2¶

commit(Msg, WalletOrOpts) -> any()

Sign a message with the given wallet.

commit/3¶

commit(Msg, Wallet, Format) -> any()

commitment/2¶

commitment(Committer, Msg) -> any()

Extract a commitment from a message given a committer ID, or a spec message to match against. Returns only the first matching commitment, or not_found.

commitment/3¶

commitment(CommitterID, Msg, Opts) -> any()

commitment_devices/2¶

commitment_devices(Msg, Opts) -> any()

Return the devices for which there are commitments on a message.

committed/3¶

committed(Msg, List, Opts) -> any()

Return the list of committed keys from a message.

conversion_spec_to_req/2 *¶

conversion_spec_to_req(Spec, Opts) -> any()

Get a codec device and request params from the given conversion request. Expects conversion spec to either be a binary codec name, or a map with a device key and other parameters. Additionally honors the always_bundle key in the node message if present.

convert/3¶

convert(Msg, TargetFormat, Opts) -> any()

Convert a message from one format to another. Taking a message in the source format, a target format, and a set of opts. If not given, the source is assumed to be structured@1.0. Additional codecs can be added by ensuring they are part of the Opts map -- either globally, or locally for a computation.

The encoding happens in two phases: 1. Convert the message to a TABM. 2. Convert the TABM to the target format.

The conversion to a TABM is done by the structured@1.0 codec, which is always available. The conversion from a TABM is done by the target codec.

convert/4¶

convert(Msg, TargetFormat, SourceFormat, Opts) -> any()

default_tx_list/0¶

default_tx_list() -> any()

Get the ordered list of fields as AO-Core keys and default values of the tx record.

default_tx_message/0 *¶

default_tx_message() -> any()

Get the normalized fields and default values of the tx record.

filter_default_keys/1¶

filter_default_keys(Map) -> any()

Remove keys from a map that have the default values found in the tx record.

find_target/3¶

find_target(Self, Req, Opts) -> any()

Implements a standard pattern in which the target for an operation is found by looking for a target key in the request. If the target is self, or not present, the operation is performed on the original message. Otherwise, the target is expected to be a key in the message, and the operation is performed on the value of that key.

format/1¶

format(Item) -> any()

Format a message for printing, optionally taking an indentation level to start from.

format/2¶

format(Bin, Indent) -> any()

from_tabm/4 *¶

from_tabm(Msg, TargetFormat, OldPriv, Opts) -> any()

id/1¶

id(Msg) -> any()

Return the ID of a message.

id/2¶

id(Msg, Opts) -> any()

id/3¶

id(Msg, RawCommitters, Opts) -> any()

is_signed_key/3¶

is_signed_key(Key, Msg, Opts) -> any()

Determine whether a specific key is part of a message's commitments.

match/2¶

match(Map1, Map2) -> any()

Check if two maps match, including recursively checking nested maps. Takes an optional mode argument to control the matching behavior: strict: All keys in both maps be present and match. only_present: Only present keys in both maps must match. primary: Only the primary map's keys must be present. Returns true or {ErrType, Err}.

match/3¶

match(Map1, Map2, Mode) -> any()

match/4¶

match(Map1, Map2, Mode, Opts) -> any()

matchable_keys/1 *¶

matchable_keys(Map) -> any()

minimize/1¶

minimize(Msg) -> any()

Remove keys from the map that can be regenerated. Optionally takes an additional list of keys to include in the minimization.

minimize/2 *¶

minimize(RawVal, ExtraKeys) -> any()

normalize/2 *¶

normalize(Map, Opts) -> any()

Return a map with only the keys that necessary, without those that can be regenerated.

print/1¶

print(Msg) -> any()

Pretty-print a message.

print/2 *¶

print(Msg, Indent) -> any()

restore_priv/3 *¶

restore_priv(Msg, EmptyPriv, Opts) -> any()

Add the existing priv sub-map back to a converted message, honoring any existing priv sub-map that may already be present.

signers/2¶

signers(Msg, Opts) -> any()

Return all of the committers on a message that have 'normal', 256 bit, addresses.

to_tabm/3 *¶

to_tabm(Msg, SourceFormat, Opts) -> any()

type/1¶

type(TX) -> any()

Return the type of an encoded message.

uncommitted/1¶

uncommitted(Msg) -> any()

Return the unsigned version of a message in AO-Core format.

uncommitted/2¶

uncommitted(Bin, Opts) -> any()

unsafe_match/5 *¶

unsafe_match(Map1, Map2, Mode, Path, Opts) -> any()

verify/1¶

verify(Msg) -> any()

wrapper function to verify a message.

verify/2¶

verify(Msg, Committers) -> any()

verify/3¶

verify(Msg, Committers, Opts) -> any()

with_commitments/3¶

with_commitments(Spec, Msg, Opts) -> any()

Filter messages that do not match the 'spec' given. The underlying match is performed in the only_present mode, such that match specifications only need to specify the keys that must be present.

with_only_committed/2¶

with_only_committed(Msg, Opts) -> any()

Return a message with only the committed keys. If no commitments are present, the message is returned unchanged. This means that you need to check if the message is: - Committed - Verifies ...before using the output of this function as the 'canonical' message. This is such that expensive operations like signature verification are not performed unless necessary.

with_only_committers/2¶

with_only_committers(Msg, Committers) -> any()

Return the message with only the specified committers attached.

with_only_committers/3¶

with_only_committers(Msg, Committers, Opts) -> any()

without_commitments/3¶

without_commitments(Spec, Msg, Opts) -> any()

Filter messages that match the 'spec' given. Inverts the with_commitments/2 function, such that only messages that do not match the spec are returned.

Module hb_message.erl¶