Construction of evaluation index system for overall training of troops

Stone Path, Red Litchi

【Abstract】This paper investigates the construction of an overall training evaluation index system for military units, aiming to enhance their capability to execute military tasks. It first elaborates on the concept of overall training and the necessity of constructing an evaluation index system. Subsequently, it details the three main components of the evaluation index system: index hierarchy, evaluation criteria, and index weighting. In the section on index weighting, it distinguishes between subjective and objective weighting methods, and provides an in-depth analysis of the principles, steps, and advantages and disadvantages of each method. Subjective weighting methods include the Delphi method, Analytic Hierarchy Process (AHP), binomial coefficient method, and chain ratio scoring method; objective weighting methods include the coefficient of variation method, entropy weight method, principal component analysis, factor analysis, and grey relational analysis method.

【Keywords】Training Evaluation | Index System | Index Weighting

Overall training for military units is the coordinated training of various elements aimed at enhancing their ability to carry out combat missions. The core of this training lies in improving the coordination level of different departments according to their responsibilities and the operational process, thereby forming an integrated combat capability. To enhance the quality and effectiveness of overall training, the key is to establish an assessment and evaluation index system that aligns with actual combat requirements. This system lays the foundation for comprehensive, systematic, and precise overall training assessments, ensuring that the evaluation results can thoroughly test the unit's combat capabilities and promote the continuous improvement of overall training levels and the steady enhancement of core combat power.

I. Division of Levels for Overall Training Evaluation Indicators

Since each indicator is set around the evaluation purpose, with strong directivity and consistency, a method of decomposing objectives layer by layer can be used to establish the structure of the indicator system. To accurately reflect the overall training comprehensive situation, the evaluation indicator system usually should be divided into three to four levels of indicators. Generally, the overall training's total objective task is taken as the "root" and serves as the first-level indicator; major evaluation project categories are derived from this "root" to form the second-level indicators; each second-level indicator is further broken down into evaluation projects to form the third-level indicators; specific evaluation contents are divided based on the third-level indicators, and so on, gradually refining the details.

2. Establishment of Overall Training Evaluation Metrics and Standards

Evaluation criteria are the yardsticks for assessing the quality of each indicator. This is the most important, complex, and scientifically rigorous task in establishing an evaluation indicator system. If the evaluation criteria for indicators are not scientifically or clearly set, the established indicator system will be difficult to function effectively. Since each level of indicators has a clear subordinate relationship, the evaluation criteria directly affect only the terminal indicators. Each terminal indicator may have one evaluation criterion or several evaluation criteria.

III. Weighting Method for Overall Training Evaluation Indicators

Index weighting is the process of determining the relative importance of each evaluation indicator within the overall evaluation system based on different criteria or information sources. Index weighting methods generally fall into two categories: subjective weighting methods and objective weighting methods.

(1) Subjective Weighting Method

Subjective weighting methods are approaches used in the construction of evaluation systems where the weights of various indicators are determined based on the decision-maker's subjective judgment, expert experience, intuition, and other non-quantitative information. The advantages of this method lie in its ability to fully consider the decision-maker's professional knowledge and specific needs in particular situations, especially when data is difficult to quantify or objective data is unavailable, providing strong application flexibility. However, the disadvantages include being subject to personal biases, cognitive limitations, and other factors, which may lead to significant subjectivity and uncertainty in the determination of weights. Common subjective weighting methods include the following:

Delphi Method

The Delphi Method, also known as the Expert Survey Method, involves selecting a group of experts to form an evaluation panel. Each expert independently provides a set of weights, forming an evaluation matrix, from which a comprehensive set of weights is derived through comprehensive processing. This method leverages the experts' knowledge, wisdom, and experience, which are unquantifiable and inherently ambiguous, to form evaluation weights for various aspects, reflecting the subjective preferences of the evaluators. The advantages of this method include its simple operation, clear principles, and the ability to fully reflect the decision-making orientation of the training organizers and leaders. However, the weights are significantly influenced by subjective factors, making it difficult to establish a set of weights that are both persuasive and stable. This method is suitable for evaluation projects where data collection is challenging or where information is difficult to quantify accurately.

Analytic Hierarchy Process

Analytic Hierarchy Process (AHP) is a method that, based on the nature of the research object, decomposes the goal to be achieved into multiple constituent elements, organizes them hierarchically according to the dominance relationships between the elements, and forms an orderly hierarchical structure model. By solving the eigenvector of the judgment matrix, it determines the priority weights of each element relative to the element at the next higher level.

Binomial Coefficient Method

The binomial coefficient method is commonly used to handle index systems that exhibit symmetry or orderliness. For a system composed of n indices, each index is treated as a term in a binomial, with its weight determined by the binomial coefficient. The key to the binomial coefficient method lies in calculating the weights using the binomial theorem from combinatorial mathematics. This method is suitable for situations where the indices exhibit a certain degree of symmetry; it is not applicable if the indices differ significantly. It is important to note that, despite its name as the "binomial coefficient method," the specifics of how to accurately map binomial coefficients to weight assignments may vary depending on the specific application scenario.

Sequential Scoring Method

The sequential grading method, also known as the adjacent comparison method, is a subjective weighting method that gradually allocates weights by comparing the importance of adjacent indicators step by step. The basic steps are as follows:

Initial Rating: For all indicators in the evaluation system, experts or decision-makers need to conduct an initial overall importance rating or ranking.

Adjacent Comparison: Conduct a comparison and scoring of adjacent indicators from top to bottom (or from the most important to relatively less important), that is, experts sequentially compare the importance of the first indicator with the second, the second with the third, and so on.

Temporary Assignment and Adjustment of Weights: The result of comparison between each pair of adjacent indicators is converted into a score or ratio, temporarily designated as the importance coefficient of these two indicators. To ensure the continuity and stability of weight distribution, these temporary coefficients are often adjusted.

The sequential comparison method is simple and easy to operate, particularly suitable for situations where the number of evaluation indicators is limited and experts can clearly compare the importance differences between adjacent indicators. Through recursive comparison and correction, this method can reduce biases caused by single weighting decisions to some extent, making it closer to actual evaluation needs.

(II) Objective Weighting Method

Objective weighting methods refer to approaches in constructing evaluation index systems that determine the weights of various indicators based on the inherent characteristics, statistical laws, or correlations of the data itself, rather than relying entirely on subjective judgments from experts or decision-makers. These methods emphasize data-driven and objective factual foundations and are commonly used in scenarios with a large amount of historical or statistical data available for analysis. Below are several common objective weighting methods:

Coefficient of Variation Method

The coefficient of variation method, based on statistical principles, is used to evaluate the degree of data variability among various indicators within an index system. It is suitable for calculating the weights of indicators when the units or dimensions of multiple indicators are inconsistent, or when it is desired to emphasize indicators with larger fluctuations.

The coefficient of variation is the ratio of the standard deviation to the mean, and the formula is as follows:

(Coefficient of Variation)/

where: is the standard deviation of the indicator, which reflects the degree of dispersion of the data around the mean; is the mean of the indicator.

In the process of empowerment, the specific steps of the coefficient of variation method are as follows:

Calculate the standard deviation and mean for each evaluation metric;

Calculate the coefficient of variation for each indicator according to the formula above.

Sum the coefficients of variation of each indicator to obtain the total coefficient of variation;

Calculate the weight of each indicator. Calculation formula: Weight coefficient of variation / Sum of total coefficient of variation.

Entropy Weight Method

The entropy weight method () is an objective weighting method based on the theory of information entropy. Entropy in information theory represents the uncertainty of information; the greater the entropy, the higher the uncertainty of the information and the more new information it contains. In an evaluation index system, the entropy weight method posits that the greater the new information provided by an index, the higher its weight should be. The specific steps for assigning weights using the entropy weight method are as follows:

Data Preprocessing: For each evaluation metric, first collect a group of sample data, and then standardize or normalize it;

Calculate the probability distribution: Compute the probability of each indicator's different values appearing across all samples;

Calculate the entropy of indicators: Use the information entropy formula to calculate the entropy value for each indicator, with the formula being: -∑(). Here, represents all possible values of the indicator, and is the logarithm base , commonly replaced by the natural logarithm.

Through the entropy weight method, the uncertainty of subjective weighting can be effectively avoided, and the information value of each indicator in the overall evaluation system can be more objectively reflected. However, the entropy weight method also relies on the quality and completeness of data. In practical applications, the effectiveness of the entropy weight method may be affected by data that is either highly concentrated or highly dispersed.

(III) Combination Weighting Method

Due to the advantages and disadvantages of subjective weighting methods and objective weighting methods, a combination of both is often employed in practical applications to enhance the stability and reliability of the weighting results.

Arithmetic Mean Method

The most straightforward combination method, which involves simply averaging the subjective and objective weights to obtain the final weights, is as follows:

Wfinal=（Wsubjective+Wobjective）/2

Among them, the weights obtained through subjective methods, and the weights obtained through objective methods.

Weighted Average Method

According to the credibility or importance given to subjective and objective weights, different weights are assigned, followed by a weighted average. The formula is as follows:

Wfinal=Wsubjective×α+Wobjective×(1−α)

Among them, it is the weight coefficient of subjective weights, with a value range between and .

Product Averaging Method

Obtain the combined weight by calculating the average of the products of two weights, as follows:

Calculate the combined weights for each indicator:

Wfinal,i=∑(Wsubjective,i×Wobjective,i)/n

To ensure that the sum of the portfolio weights equals 1, it is necessary to normalize the calculated portfolio weights:

Wnormalized,i=Wfinal,i/∑Wfinal,i

IV. Conclusion

In summary, constructing a scientifically and rationally comprehensive training evaluation index system is crucial for enhancing the combat capabilities and training quality of the troops. By comprehensively applying subjective weighting and objective weighting methods, we can more thoroughly assess the overall training effectiveness and provide strong support for decision-making. However, each weighting method has its limitations, so in practice, different weighting methods should be flexibly selected and combined based on factors such as the evaluation purpose, data availability, and expert opinions.

references

Military Terms of the Chinese People's Liberation Army..

Du Liping, Cai Erqing. Research on Basic Issues of Military Training Evaluation. Military Operations Research and Systems Engineering, ().

Zhang Jie, Tang Hong. Research on Performance Evaluation Methods. Beijing: National Defense Industry Press.

Liu Zhiqiang, Wang Huiqing, Wen Ying. Overview and Brief Review of Technology Assessment Methods. Technology and Innovation, ().

Yang Yu. Analysis of Weighting Methods in Multi-Index Comprehensive Evaluation. Theoretical Exploration, ().

Enter the mini program and give a thumbs-up to the papers you like!

Editor of this issue: Chen Luyang

This issue reviewed by: Lin Yingxi

Layout Design: Zhou Kaikai

Article Source: "China Military-to-Civilian Magazine"

Collaboration Email: .

Submission email: .