Use `collect` to decode `Vec`.

It's hyper-optimized, we don't need our own unsafe code here.

This requires getting rid of all the `Allocator` stuff, which isn't
needed anyway.