R&D
TSIP R&D division develops and proposes cutting-edge technology solutions in AI/ML.
We overview our “Data Analytics” and “Computer Vision” research activities.
Data Analytics
Low Visibility Forecasting Research
The R&D Division of Toshiba Software (India) Pvt. Ltd., in collaboration with the Corporate R&D Center of Toshiba Corporation, has developed machine-learning approaches for forecasting low visibility caused by fog, rain, and snow. Accurate low-visibility forecasting is critical for the safety and operation of road, sea, and air traffic. The two publications below present successive advances in this line of work.
Forecasting of Low Visibility Using Weather and Air Quality Data
Low visibility caused by fog, mist, haze, smoke, rain, or snow affects the safety and operation of road, sea, and air traffic, sometimes leading to serious accidents. Conventional time-series forecasting using weather and air quality data struggles to predict low visibility accurately because the visibility data is both imbalanced and clipped. Our approach combines two forecasting models — one using kernel density estimation (KDE)-based weights to handle clipped data and one using discrete weights to handle imbalanced data — and fuses their outputs through a probabilistic classifier (logistic regression) that estimates the likelihood of the low and high visibility ranges.
Figure 1: Outline of the proposed approach (KDE- and discrete-weight models combined via a probabilistic classifier).
Validated on four publicly available datasets from the USA (California, Texas, Washington DC) and Japan (Urakawa), the approach improves RMSE and MAE in the low visibility range by 29–54% and 40–59% respectively, and improves the F-score of low/high visibility forecasting by 0.20–0.26 on average over existing technologies for 3–24 hour ahead forecasting. Relative humidity was found to be the most important factor affecting visibility, with wind direction a further key weather factor.
Figure 2: 3-hour ahead forecasting on the California dataset.
Figure 3: 3-hour ahead forecasting on the Texas dataset.
Publication: T. Paul, V. Reddy, S. P. K. Ayyagari, R. Shingaki and K. Nishino, “Forecasting of Low Visibility Using Weather and Air Quality Data for Safe and Smooth Transportation Operation,” 2024 23rd IEEE International Conference on Machine Learning and Applications (ICMLA), 2024, doi: 10.1109/ICMLA61862.2024.10903276.
Low Visibility Forecasting Using Numerical Weather Prediction Data
Building on the earlier work, this approach incorporates Numerical Weather Prediction (NWP) data to forecast low visibility. Conventional methods using time-series visibility and meteorological data struggle with data imbalance and censored data, degrading accuracy in the low visibility range. We employ a Censored Quantile Regression Neural Network (CQRNN) to handle censored data and a Light Gradient-Boosting Machine with KDE weights to handle imbalanced data, then combine their forecasts using a logistic regression probabilistic classifier.
Figure 4: Outline of the proposed approach — CQRNN and LightGBM+KDE models ensembled via a probability model.
The CQRNN model estimates conditional quantiles of visibility, providing a data-driven, lower-bound estimate for the rare low-visibility events, replacing the manually defined discrete weights of the earlier method.
Figure 5: Illustration of forecasting by CQRNN at various quantiles.
Validated on observed and Meso-Scale Model (MSM) forecast data from the Japan Meteorological Agency at the Sendai and Akita stations, the approach forecasts low visibility with high accuracy up to 24 hours ahead. Adding NWP forecast data significantly improves performance by mitigating the peak-shift problem.
Figure 6: Forecasting using observed and forecast meteorological variables (3-hour and 24-hour rolling forecast).
The approach was further evaluated under specific weather conditions that cause low visibility — fog, snow, and rain — confirming accurate short-term forecasts across all three.
Figure 7: Forecasting of low visibility caused by fog (3-hour and 24-hour rolling forecast).
Figure 8: Forecasting of low visibility caused by snow.
Figure 9: Forecasting of low visibility caused by rain.
Publication: T. Paul, V. Reddy, S. Prem Kumar Ayyagari, R. Shingaki, K. Nishino and Y. Shiga, “Low Visibility Forecasting Using Numerical Weather Prediction Data,” in Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track and Demo Track (ECML PKDD 2025), Lecture Notes in Computer Science, vol. 16022, Springer, Cham, 2026, pp. 196–212, doi: 10.1007/978-3-032-06129-4_12.
Residual Signature Analysis for Wind Turbine Failure Prognosis
To improve the operation and maintenance of wind turbines, our approach revolutionizes failure prognosis beyond conventional methods. The proposed method advances the capabilities of existing residual analyses inspired by electrical signature analysis used in motor failure detection.
- Residual Generation - Residual generation is crucial for accurate performance analysis in systems exposed to environmental factors, as it eliminates the effects of external variables, such as seasonal changes, on the data being examined. This ensures that evaluations are based on the intrinsic behavior of the system rather than external influences.
- Signature Generation - This step enhances the detection of failure signs with a precise temporal resolution, allowing for more accurate monitoring. This process effectively manages the fluctuation of residual errors, reducing noise and improving the reliability of failure detection.
- Anomaly Score Calculation - The anomaly score is calculated as the mean squared error (MSE) distance from a segment’s signature to the nearest centroid, accounting for variations due to aging or different operational conditions.
- Anomaly Detection - To identify anomalies, control charts are used to signal when a process strays from its normal state. These charts may feature variable or fixed limits, including an Upper Control Limit (UCL) and a Lower Control Limit (LCL), which are determined based on the anomaly scores from the training data.
This innovative approach offers a scalable and efficient solution to extend the lifespan and improve the safety of wind energy operations.
Publication: T. Paul, V. Reddy, S. P. K. Ayyagari and K. Nishino, "Residual Signature Analysis for Wind Turbine Failure Prognosis," 2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE), Grenoble, France, 2023, pp. 1-5, doi: 10.1109/ISGTEUROPE56780.2023.10407545.
Optimization solution: Boiler Combustion Optimization (BCO)
- We can improve the power plant efficiency by optimizing operational timing for boiler combustion & cleaning, which is calculated based on the machine learning method. The service architecture is simple & scalable and can be applied to an existing power plant system.
- Boiler Combustion Optimization(BCO) solution for power plant system was registered on Toshiba SPINEX Marketplace.
Toshiba SPINEX Marketplace: Boiler Operation Optimization
https://www.spinex-marketplace.toshiba/en/services/boiler-operation
*This project is a collaboration work with Toshiba Corporation and sponsored by New Energy and Industrial Technology Development Organization, Japan
Edge AI technology
- IoT systems are the source of generating a huge amount of time series data. Deep neural networks and advanced machine learning algorithms provide precise forecasting using generated time series data. But, it requires computationally expensive infrastructure like cloud computing and GPU machines. Cloud computing is not feasible in the time-critical decision-making process in the factory. The decision must be made for newly generated input data.
- Edge computing is the solution for faster responses. However, edge devices have limited memory and processing power capabilities. Incremental learning is the solution used in Edge devices for a speedy response without compromising the prediction accuracies. We proposed ILVAR (Incremental Learning Vector Auto Regression for Forecasting with Edge Devices) for time sequential signal forecasting and published it in IEEE ICMLA.
Pesala, V., Paul, T.K., Ueno, K., Bugata, H.G., & Kesarwani, A. (2021). Incremental Learning Vector Auto Regression for Forecasting with Edge Devices. 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), 1153-1159.
Figure: Incremental Learning at the Edge
Computer Vision
We will assist the industry in achieving its tasks utilizing machine learning method for images and videos. The industry problems range from classification, segmentation, analysis, quality measurements, anomaly detection, and image generation. In most cases, more data or the lack of data for training is challenging. We are trying to harness existing methods and developing novel methods to achieve the goal.
AI steel grading
- Human professional comparable steel quality grading service was registered to Toshiba SPINEX Marketplace (METALSPECTOR/AI).
- We can estimate steel grades from micrographs of the steel surface. We improve the estimation accuracy utilizing multiple deep learning methods and increasing the number of image data for dictionary learning.
TOSHIBA SPINEX Marketplace: METALSPECTORtm/AI
https://www.spinex-marketplace.toshiba/ja/services/metalspector-ai
TOSHIBA SATLYS Solutions: METALSPECTORtm/AI
* This project is a collaboration work with Toshiba Digital Solutions Corporation
Domain Adaptation
- Most real-time applications have fewer data (target data) to train deep learning-based methods. Directly utilizing existing methods trained on other kinds of data (source data) does not yield satisfactory results due to a significant domain gap between target and source data.
- We propose unsupervised domain adaptation techniques to mitigate issues like domain gaps. This method uses label-rich source domain information to generate labels for the target domain and then refine it to generate more accurate labels for the target domain.
Maurya, J., Ranipa, K. R., Yamaguchi, O., Shibata, T., & Kobayashi, D. (2023). Domain Adaptation Using Self-Training With Mixup for One-Stage Object Detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 4189-4198).
Figure: Method to utilize label-rich source domain data knowledge to generate labels to non-labelled target domain
