Scalable Data Science and Artificial Intelligence Frameworks for Real Time Big Data Processing and Operational Optimization

  • Home/
  • Scalable Data Science and Artificial Intelligence Frameworks for Real Time Big Data Processing and Operational Optimization

Scalable Data Science and Artificial Intelligence Frameworks for Real Time Big Data Processing and Operational Optimization

Okoye, Johnson C.1, Emekwisia, Chukwudubem C.2*, Odubunmi, Oluwafikunmi M.3, Nganji, Christopher E.4, Agweven, Philomena E.5, Akinbamilowo, Oladimeji O.6

1Department of Engineering and Technology Management, Louisiana Tech University, Ruston LA, USA
2Department of Metallurgical and Materials Engineering, Nnamdi Azikiwe University, Awka, Nigeria
3Department of Computer Science, Babcock University, Ilisha-Remo, Nigeria
4Department of Petroleum Engineering, Federal University of Petroleum Resource, Effurun, Nigeria
5Department of Logistics and Global Operations, University of Lincoln, United Kingdom
6Department of Metallurgical and Materials Engineering, Federal University of Technology, Akure, Nigeria

ABSTRACT

The proliferation of big data and real-time analytics has necessitated the development of scalable frameworks for data science and artificial intelligence (AI). This research aims to design and evaluate a scalable AI-based architecture capable of real-time processing and operational optimization across multiple domains. Using Apache Spark, Kafka, and TensorFlow, we implemented a streaming data pipeline for predictive analytics in industrial IoT and financial transaction environments. Results showed a 42% reduction in latency (from 1.2s to 0.7s) and a 37% increase in throughput (from 4200 to 5750 records/sec). Figures 1 and 2 illustrate the improvements in system performance. The framework demonstrates practical applicability in sectors requiring fast, scalable, and intelligent data-driven decision-making, such as manufacturing, cybersecurity, and digital finance.

Keywords: Big Data Analytics; Real-Time Processing; Artificial Intelligence; Scalability; Operational Optimization

IJISE 9.1 (1-6)Download

Citations – APA

Okoye, J. C., Emekwesia, C. C., Odubunmi, O. M., Nganji, C. E., Agweven, P. E., & Akinbamilowo, O. O. (2025). Scalable Data Science and Artificial Intelligence Frameworks for Real Time Big Data Processing and Operational Optimization. International Journal of Information Sciences and Engineering, 9(1), 1-6. DOI: https://doi.org/10.5281/zenodo.17091292

Categories: