The increasing use of public clouds for high-performance computing (HPC) demands robust security and privacy mechanism for data-in-use. Although the principles of a Zero-Trust security model are reputable, their practical implementation as a privacy-preserving architecture in container-based frameworks for HPC remains largely unfamiliar. This research investigates a secure framework that utilizes Apptainer with Microsoft Azure's Confidential Computing to protect a compute-intensive matrix multiplication task. We evaluate the performance implications of continuous memory encryption, process level attestation, and secure system calls. Our findings reveal measurable latency at the virtualization and hardware layers, emphasizing the overhead as necessary to achieve verifiable data privacy. Furthermore, this study establishes the practical capability of a security architecture that leverages Trusted Execution Environments (TEEs) to simultaneously support data confidentiality and integrity during computation. By conducting empirical evaluations on representative HPC workloads, the research measures the performance overhead resulting from the secure execution. The findings gathered offer critical insights into the trade-offs between privacy enforcement and computational efficiency. Additionally, this groundwork serves as a baseline for adopting a privacy-first Zero Trust framework in public cloud-based high-performance computing environments.