[4] Accuracy of Mind Health Measurement System

The mind health measurement system is not only simple, but also has high discrimination power.

Accuracy

MiHMS uses “MIMOSYS(*1)” as a voice analysis engine. “MIMOSYS” extracts emotional parameters from voice and synthesizes those parameters to calculate a mental health (Mind Activity).

Mind Activity can correctly identify a person who has a bad mental health (sensitivity is 100%). On the other hand, of those who have a good mental health, less than 40% are mistakenly identified to be sick (specificity is 61%). This is a calculation regarding discrimination accuracy. For example, if there are 100 users, a calculation is shown in the following table.

Condition: BadCondition: GoodTotal
Judge (is Bad)67 (a)13 (b)80 (a)+(b)
Judge (is Good)0 (c)20 (d)20 (c)+(d)
Total67 (a)+(c)33 (b)+(d)100

Sensitivity: (a) / ((a)+(c)) ⇒ 67 ÷ 67 = 1.0 (100%)
Specificity: (d) / ((b)+(d)) ⇒ 20 / 33 = 0.61 (61%)

How much discrimination power is required in practice?

Given the above discrimination accuracy, we consider whether the mind health measurement system can be used in practice. Since the sensitivity is 100%, it is thought that people with bad mental health can almost certainly be captured (* 3). On the other hand, with a probability of less than 40%, there is still a risk that a person with good mental health will be mistakenly determined to be a bad mental health. 

However, this risk is only the probability of misclassification per measurement. “MiHMS” mitigates a misclassification risk by increasing the number of results and checking the trend of results using an advantage of easy high frequency measurement. In addition, misclassification risk can also be mitigated by using risk management methods such as rating system.

It is difficult to determine a definite level for the discrimination power in practice, but (1) risks can be controlled and (2) business design can be done are important as a point to judge the validity. As this point, “MiHMS” has a reasonable discrimination power.

*1 It means the Mind Monitoring System developed by PST Inc., a technology company that visualizes the state of the mind and body.
*2 Tokuno et al (2017), “CASE STUDIES OF UTILIZATION OF THE MIND MONITORING SYSTEM (MIMOSYS) USING VOICE AND ITS FUTURE PROSPECTS”
*3 The term “almost” is used because it is not possible to completely eliminate factors that affect accuracy, such as differences in measurement conditions, individual differences, and deterioration over time.
*4 The AUC is 0.81. We think that this is a reasonable level in practice.

Corporate profile

Risk Measurement Technologies Co., Ltd.
Corporate sites: https://rimtech.co.jp/index.php/about-us/

Contact us

Please contact us from the following URL.
URL: https://rimtech.co.jp/index.php/inquires/

Follow me!