ComponentType: add as_val method to convert to tagged representation

[pchickey]

This PR adds the method fn to_val<S>(&self, store: StoreContextMut<S>) -> Result<Val> to wasmtime's ComponentType trait. This is a first step in a bigger project to provide better debugging facilities for wasmtime component model execution.

Context

Wasmtime separates component model values into two separate worlds:

• a typed representation, where Rust datatypes corresponding to component model types derive or impl the ComponentType trait, plus optionally the Lift and Lower traits (required to be used in arguments and return values of import functions, respectively) which provide wasmtime the means to convert to/from each Rust type and its canonical ABI representations. To use this representation for an import function wasmtime users use wasmtime::component::Linker::func_wrap and friends, and on an export function use wasmtime::component::Instance::typed_func and friends.
• a tagged representation, where all component model types are represented using the different variants of the wasmtime::component::Val enum. To use this representation for an import function, wasmtime users use wasmtime::component::Linker::func_new and on an export wasmtime::component::Instance::func.

Prior to this PR, users have to choose whether to opt into the typed representation or the tagged representation based on what their application needs, and we (the wasmtime developers) focused most of our efforts on the typed representation because that ends up being the right choice for most embedders.

Users of the typed representation do so almost exclusively through the use of wasmtime::component::bindgen!, a proc macro which generates Rust structs and enums corresponding to component model types it parses from wit, and delegates to the #[derive(ComponentType, Lift, Lower)] derive proc-macros in those generated types (which themselves also are provided by the wasmtime crate). This bindings generation typically takes place in library code, e.g. wasmtime-wasi, wasmtime-wasi-http and related crates, and bindgen takes a variety of options that influence exactly what sort of Rust representation to use for wit types - the ComponentType derive macros permit flexibility in various ways. But, at any rate, to my knowledge every "serious" embedding of wasmtime (rough definition: engineers ship it to prod at dayjob) is written in Rust and uses bindgen exclusively or very close to it.

On the other hand, the tagged representation is useful for library code that exists independently from a given wit description. The library for a human-friendly serialization format for component model values https://crates.io/crates/wasm-wave exports some traits which wasmtime provides impls for in terms of wasmtime::component::Val. Wave ends up being useful for features in wasmtime-cli like wasmtime run --invoke ..., which I introduced in #10054. The C API, which is used for bindings to other languages, also uses the Val representation, because C APIs and the bindgen proc macro are oil and water.

My goal is to create better debugging tools for wasmtime component model programs, providing wasmtime embedders and cli users with the capability to inspect, and eventually record/replay or modify component model values. Like the wave crate and invoke integration, debugging tools benefit from being independent of a given wit description. Debugging tools must integrate with wasmtime embeddings regardless of whether they use the typed or tagged representation to implement import funcs and call export funcs, and since most embeddings are making substantial use of bindgen! in public and private crates against public and private wits, my approach is to provide a bridge where a debugging library can operate on just the tagged representation of values, using a new mechanism built into all typed representations to convert to tagged representation. Further mechanisms in wasmtime to hook into library code using the typed representation will come in future PRs, see #12532 for work in progress.

Converting typed to tagged

For every value in a typed component model value representation (i.e. any value of a type which has an impl ComponentType), there also exists an equivalent component model value in wasmtime::component::Val (tagged) representation. Prior to this PR, there was no mechanism to convert a typed value to a tagged one.

This PR adds a method to_val to ComponentType to convert to a Val:

fn to_val<S>(&self, store: StoreContextMut<S>) -> Result<Val>

The design choices in this signature are:

• The method takes &self, not self, because the reverse conversion does not exist, and in the context of adding debugging hooks to existing code using the typed representation, we don't want to destroy the typed representation in order to observe it.
• The method gives the owned representation Val, which means making a structural clone of the value. For a wit list<u8> this ends up having some fairly ugly overhead: the typical Vec<u8>, which impls ComponentType, Lift, Lower, converts to the Val::List(Vec<Val>) variant, where each vec element is then a Val::U8(u8) variant. Other variants, like Val::Record, Val::Variant, Val::Enum, and Val::Flags end up using String representations of the various wit identifiers used for record rows, variant and enum tags, and each flag value. This requires substantially more memory to represent, and is substantially less efficient to traverse!
• The owned representation of Val has further caveats where it does not provide the same safety guarantees as the typed representation for resource, future, and stream representations, see https://docs.rs/wasmtime/latest/wasmtime/component/struct.ResourceAny.html for information on the requirements for dropping a resource, which also apply when in a leaf node of a Val. When Resource<T>::to_val(&self) is invoked, FIXME FIGURE OUT WHAT HAPPENS HERE AND WHAT THE USER NEEDS TO DO TO HANDLE IT. DOES VAL NEED A drop{,_async}(self,StoreContextMut) METHOD? and put whatever i learn in the proper docs
• The conversion requires a store. I originally used impl AsContextMut for the store, but Alex suggested I switch to use StoreContextMut, which then requires the function to be generic on the type in the store. For this type variable I use S instead of the usual T because T is often already in use in the impl.
• The conversion is fallible. The conversion can fail if you use the wrong store for a resource type. Soon, when Nick gets around to changing the Val representation to the new fallible allocation types, it will also be fallible due to OOM, so its nice that the general shape of things is there.

The ComponentType derive macro fills out a definition of to_val mechanically - this was the only aspect of the PR that was subtle to figure out. All of the impl ComponentType for Rust types provided by wasmtime get a straightforward mechanical implementation.

TODO before this PR can land:

• literally any testing at all. im confident that at the time of writing not a single invocation of to_val has ever been executed. whatever it typechecked
• rust docs need better
• Alex gets another pass at the design, which his feedback already substantially improved.

[alexcrichton]

Personally I'm not overly thrilled about extending the Lift/Lower/etc traits given the complexity of testing them and the sheer breadth of impls required, but this is overall relatively minor in terms of complexity. Before landing mind also expanding the PR description with some rationale for the expected use cases?

Bikeshedding wise I might say to_val instead of as_val since "as" implies "cheap" typically and this definitely isn't cheap. I'd also bikeshed the signature a bit to use StoreContextMut<T> rather than impl AsContextMut to avoid the extra indirection of impl Trait where possible.

The 'static bounds popping up on futures/streams is fine, those are only really practically usable with 'static types anyway, so I don't think that'll be an issue. The specifics of as_val for futures/streams I think will want to be re-worked a bit too to avoid manually creating structs, but I can take a closer look once this is closer to landing.

Similar to futures/streams I'll need to think a bit harder about the try_as_resource_any method added here for resources. I forget exactly where we are on clone/copy/etc on those and how it affects what we document in terms of guarantees and whatnot.