Fix memory leaks and add checks for failing malloc() calls to testcapi module

CID 1058288