Healthcare Big Data Analytics Platform with Hadoop/MapReduce Framework

Big data analytics (BDA) is important to reduce healthcare costs. However in many hospital systems new technologies that influence patient data require extensive technical and rigorous usability testing before implementation into production. Therefore to implement an existing High Performance Computing (HPC) Linux node clusters were utilized externally and simulation of patient data benchmarked and cross-referenced with current metadata profiles in operational hospital systems at the Vancouver Island Health Authority (VIHA) Victoria Canada. Over the tested platform the data were generated indexed and stored over a Hadoop Distributed File System (HDFS) to noSQL database (HBase) that represented three billion patient records. Hadoop/MapReduce framework formed the BDA platform with HBase (NoSQL database) using hospital-specific metadata and file ingestion. Queries showed high performance with a variety of Apache tools in Hadoop’s ecosystem. BDA platform of HBase distributed by Hadoop successfully under high performance at large volumes and the entire data archive. Importance on representation of health informatics using technologies for big data in healthcare is discussed.