August 2023 Volume 5

FORGING RESEARCH

was imaged and the regions containing scale are indicated by the squares superimposed on the processed image.

quantitative tests are routinely employed for this purpose. In the present study, a confusion matrix was used to analyze the accuracy of the complete fill/incomplete fill predictions made by the ANN. The so-called “confusion matrix” is a predictive analytics tool that is widely used in Al to analyze how a machine learning classifier performed on a given dataset. In simple terms, it displays and compares actual values with the ANN model's predicted values in a tabular format. A stratified 5-fold cross validation was used to evaluate the effectiveness of an ANN classifier. A stratified 5-fold cross validation partitions the data into five parts that each have a similar number of complete fill/incomplete fill parts. Five data sets are formed with a different partition acting as the test set. For the discrete input ANN, there are 809 true positives (actual complete fill correctly predicted as complete fill) and 1407 true negatives (actual incomplete fill correctly predicted as incomplete fill). The ANN classifiers (Table 2) made 64 mistakes, of which 22 actual fills were predicted as incomplete fills (false negative), and 42 actual incomplete fills predicted as fills (false positive). Table 2. Confusion matrix showing ANN predicted and actual part complete fill/incomplete fill for the forged aluminum part using discrete inputs. While the confusion matrix is a measure of the ANN’s performance, it does not provide a complete picture. Consequently, the team calculated several statistics based on the data in the confusion matrix. The accuracy, which is the number of correctly classified parts divided by the total number, is 97.2 ± 0.64%. The true positive rate, which is the number of true positives divided by the total number of positives, is 97.4 ± 2.74%. The true negative rate, which is the number of true negatives divided by the total number of negatives, is 97.1 ± 0.64%. This is a high level of agreement and confirms the ANN’s accuracy. The second ANN tested for die fill used load-stroke curves as inputs. The validation method was the same as the discrete ANN, i.e., a five-fold stratified cross validation. Our results showed that the load-stroke curves had the most profound effect as the ANN was able to achieve more than 95% accuracy of fill and incomplete fill when it was used as the sole input. The confusion matrix is shown in Table 3. The accuracy for the load-stroke ANN is 95.9 ± 1.01%, the true positive rate is 95.3 ± 2.07%, and the true negative rate is 96.3 ± 1.52%. This was somewhat lower than the discrete data points but is still very close and suggests that either approach could be useful depending on the end user’s individual needs. Execution time of both algorithms for an individual part was less than 300 ms confirming that both algorithms could be employed in real time.

Figure 5. Surface scale detection scheme. Each square indicates residual scale formation after a bust. The total execution time is only 200 milliseconds demonstrating that the system can operate in real time. Results and Discussion The performance of the die fill ANN algorithm will be considered first. Although ANN systems are considered to be “black-box”, some insights were nonetheless obtained from a qualitative evaluation of the training data. Figure 6 shows the simulated load stroke curves. A qualitative analysis of the load-stroke curves revealed several characteristics that differentiated good (complete fill) and bad (incomplete fill) parts. These include a consistent pattern of sharp transitions from upsetting to die fill, individual bands or clusters of curves that reflected temperature differences in the starting billet, and good parts that tended to have lower energy consumption. An additional observation is that using upper and lower control limits based on percentages of nominal tonnage during the press stroke may not fully capture the process behavior and fully differentiate between good/bad parts.

Figure 6. Plot showing the 2,280 simulated load-stroke curves used as training data. Parts predicted to have complete or incomplete fill conditions are indicated by blue and red curves respectively. An important consideration is how accurate the ANN predictions are based on quantitative metrics. The theoretical basis for testing ANN accuracy/predictive capability is well established and several

FIA MAGAZINE | AUGUST 2023 70