Scalable Data Science and Artificial Intelligence Frameworks for Real Time Big Data Processing and Operational Optimization
Authors: Okoye, J. C., Emekwesia, C. C., Odubunmi, O. M., Nganji, Christopher E., Agweven, P. E., Akinbamilowo, O. O.
Journal: International Journal of Information Science and Engineering (IJISE), ISSN 1694-4496
Citation: IJISE 9(1): 1-6, 2025-09-10.
DOI: 10.5281/zenodo.17091292
PDF: Download full-text PDF
Type: Original Research
Abstract
The proliferation of big data and real-time analytics has necessitated the development of scalable frameworks for data science and artificial intelligence (AI). This research aims to design and evaluate a scalable AI-based architecture capable of real-time processing and operational optimization across multiple domains. Using Apache Spark, Kafka, and TensorFlow, we implemented a streaming data pipeline for predictive analytics in industrial IoT and financial transaction environments. Results showed a 42% reduction in latency (from 1.2s to 0.7s) and a 37% increase in throughput (from 4200 to 5750 records/sec). Figures 1 and 2 illustrate the improvements in system performance. The framework demonstrates practical applicability in sectors requiring fast, scalable, and intelligent data-driven decision-making, such as manufacturing, cybersecurity, and digital finance.
Keywords
Big Data Analytics, Real-Time Processing, Artificial Intelligence, Scalability, Operational Optimization