CaloCube: An isotropic spaceborne calorimeter for high-energy cosmic rays. Optimization of the detector performance for protons and nuclei