We focus on building a new information-theoretic framework in computer vision for sensing and analysis of Big Data by upgrading recent results from homogeneous to heterogeneous data types. This include extending the compressed sensing framework towards handling prior information, structured noise and erroneous measurements, and towards applications for improved image reconstruction and content analysis in biomedical and remote sensing applications.