Detection of difficult calvings in dairy cows using neural classifier

In this study, the detection of dairy cows with difficult calvings using artificial neural networks (ANN) and classification functions (CF) is presented. The set of 15 classification variables was used. The dependent variable was the class of calving difficulty: difficult or easy. Perceptrons with one (MLP1) and two (MLP2) hidden layers as well as radial basis function (RBF) networks were analyzed. The prepared classifiers were characterized by good quality. The accuracy amounted to 75-92 %. Only the RBF network had somewhat worse quality. The level of correct detection by ANN was also high. The sensitivity on a test set was 67-80 % at specificity of 6181 %. In the case of CF, a considerable disproportion between sensitivity (6 %) and specificity (99 %) was found. The variables with the greatest contribution to the determination of calving difficulty class were calving season, CYP19-PvuII genotype, pregnancy length and, to a lesser degree, other variables. The performed analyses proved the usefulness of ANN for the detection of cows with difficult calvings, whereas the detection by CF was inaccurate.


Introduction
Cattle, through the anatomy of their pelvis, are the most predisposed to the occurrence of difficult calvings (dystocia) from among all the species of farm animals.The course of parturition itself is affected by many factors of environmental, anatomical, physiological and genetic nature and at present a large number of genes influencing the quality of calving are known.Among them the oestrogen receptor alpha gene (Szreder & Zwierzchowski 2004, Szreder et al. 2007) and aromatase cytochrome P450 gene (Vanselow et al. 1999) are mentioned.At the same time, the consequences of difficult calvings go far beyond the necessity of giving direct help by the staff and usually result in considerable economic losses.Therefore, earlier obtaining of information on the potentially difficult calving could be very advantageous to a breeder.Such a possibility is offered by, among others, artificial neural networks (ANN), which can be regarded (or at least some of their types) as artificial intelligence classics (Samarasinghe 2007).ANN have many properties that make them particularly useful for classification and detection tasks.At the same time, they do not require any a priori assumptions concerning e.g.distribution of the analysed variables or the homogeneity of variance.ANN are also insensitive to data errors and capable of capturing complex relationships among variables.Moreover, the prior specification of a model is unnecessary in this case.However, in order to verify the results obtained by means of ANN, a more traditional approach such as discriminant analysis with classification functions (CF) was used in the present study as well.Both CF and ANN have already found many applications in the husbandry and breeding of cattle.Hence, in the present study the detection abilities of the two already classical approaches (from the field of statistics and machine learning) were compared with regard to the detection of cows with difficult calvings.The combined use of the genomic, reproductive and other phenotypic factors for the detection of dystocia in dairy cattle has not been applied so far.
The detailed aims of this study were: 1) to prepare and evaluate the quality of the selected types of ANN and CF used for the determination of the calving difficulty in dairy cows, 2) to evaluate detection of animals with problems at calving by means of selected models and 3) to find which factors had the greatest influence on the course of parturition.

Material and methods
The research material consisted of 1 221 calving records of the Polish Holstein-Friesian cows of the Black-and-White strain (the mean percentage of Holstein-Friesian [HF] genes amounting to 87.73 %).The calvings occurred between 2004 and 2009.The animals were kept in a loose barn in the West Pomerania Province, where they had access to the outside run over the whole year.The mean 4 % fat-corrected milk (FCM) yield in the analysed period (305-day lactation) was 10 171 kg.The analysis included only those cows in which no serious diseases before calving were found.The description of the performance parameters is presented in Table 1.For the preparation of individual classifiers the same set of input (classification) variables was used: x 1 [%HF] percentage of HF genes in cow genotype x 2 [BGLI] oestrogen receptor alpha genotype identified using BglI restrictase (ERα-BglI) x 3 [SNA_BI] oestrogen receptor alpha genotype identified using SnaBI restrictase (ERα-SnaBI) x 4 [CYP19] aromatase cytochrome P450 genotype identified using PvuII restrictase (CYP19-PvuII) x 5 [PREG] pregnancy length (days) x 6 [BCSI] body condition score index (scores)calculated as a difference between the mean BCS for the previous production season (BCS m ) and BCS at calving (BCS): BCSI=BCS m −BCS x 7 [SEASON] calving season (1: October -May and 2: June -September, according to Klassen et al. 1990) x 8 [AGE] cow age at calving (months); x 9 [CCI] calving-to-conception interval (days) x 10 [CI] calving interval (days) x 11 [FCM] 4 % fat-corrected milk yield (kg), calculated according to the formula: FCM=0.4•ML+15•FT,where ML -real milk yield in lactation (kg), FT: fat yield (kg) x 12 [FT_PR%] the combined fat and protein content in milk (%) x 13 [FIRST_2] daily milk yield for the first two months of lactation (kg) x 14 [PEAK] daily milk yield at the peak of lactation (kg) x 15 [LAST_2] daily milk yield for the last two months of lactation (kg) The body condition of cows was assessed on a 5-point scale (Ferguson et al. 1994).The obtained scores were then modified setting the optimum at 3.50 points and at higher values deducting the multiple of 0.25 from 3.50.The cows were assessed at calving (BCS), after calving, during artificial insemination period and during pregnancy (BCS m ).The PCR-RFLP was conducted according to Vanselow et al. (1999), Szreder & Zwierzchowski (2004) and Szreder et al. (2007).For the ERα-BglI polymorphism, the AG (3 restriction fragments of 242, 182 and 60 bp) and GG (2 restriction fragments of 182 and 60 bp) genotypes were obtained.For the ERα-SnaBI polymorphism, the AA (1 restriction fragment of 340 bp without digestion) and AG (3 restriction fragments of 340, 225 and 115 bp) genotypes were found.Finally, for the CYP19-PvuII polymorphism, the following genotypes were obtained: AA (1 restriction fragment of 405 bp without digestion), AB (3 restriction fragments of 405, 327 and 78 bp) and BB (2 restriction fragments of 327 and 78 bp).
The output variable (y) was the class of calving difficulty: A -difficult calvings, with a substantial help from man and veterinarian and with potential complications such as placental abruption, stillbirth etc., B -easy calvings, including spontaneous ones or with little help from man.The percentages of calvings from both classes are presented in Figure 1.
The whole set of calving records was randomly divided (Figure 1) into 2 subsets: a set for the preparation of ANN and CF (1 016 records) and a test set for the detection of dystocia (205 records).The calvings in the test set were not used either for the building and training of neural models or for the estimation of CF coefficients.In the case of ANN (Figure 1), the set used for their preparation was additionally divided into training set (811 records) and validation set (205 records).This division was based on the proposition of Ivachnenko & Jurackovskij (1987) but, in the present study, the training set was increased by 16 % in order to provide ANN with more patterns.The validation set was created to monitor the training process, whereas the test set was used for the verification of prediction accuracy.The ANN were built and trained using Statistica Neural Networks software (StatSoft 2000).The best network was selected from among the following types: perceptrons with one hidden layer (MLP1), perceptrons with two hidden layers (MLP2) and radial basis function (RBF) networks.The search for the best network was repeated 10 times each time, which gave a total of 160 ANN for each MLP type and 80 RBF networks.
The quality of the analysed networks was determined using the root mean square (RMS) error for the training and validation sets.The error was calculated according to the following formula (Salehi et al. 1998): where y ipr is the predicted value of the output variable for the i-th calving record, y ir is the real (desired) value of the output variable for the i-th calving record and n is the total number of calving records in a given set.
The best network from each category was selected for a detailed analysis.The MLP1 and MLP2 networks were trained with the back-propagation and conjugate gradient methods until reaching the lowest possible RMS error on the validation set.In the training of RBF networks, the basis function centres were selected using k-means clustering, the width parameters were determined by means of the k-nearest neighbour algorithm and the training of the output neuron was performed with the pseudoinversion method (StatSoft 2000).For the estimation of the linear CF parameters the following formula was applied: where CF k is the classification function for easy or difficult calvings, k=1, 2, w k0 =ln q k , q kprobability that a given observation comes from the k-th population, w kj is the weight coefficient at the j-th original variable, j=1, 2, …, 15 is the number of classification variable and x 1 , x 2 , …, x 15 are the classification variables.
The value of the classification function, determined for the analysed objects, was compared with the standard, thus assigning the object to a given class.The assumptions of the CF applicability were also verified -the normality of distribution (using the W Shapiro-Wilk test) and homogeneity of variance (Levene's test).The a priori probability was determined proportionally to the size of the groups (Stanisz 2007).
The quality of classification criteria for individual variables was defined using the significance of F statistics calculated on the basis of the Wilks' lambda coefficient (Stanisz 2007): where p is the maximum number of discriminant functions, λ l is the l-th eigenvalue and i is the number of discriminant functions (i=1, 2,…).
The quality of the classification results by individual models was determined by means of sensitivity -conditional true positive probability and specificity -conditional true negative probability: where A is the number of correctly detected difficult calvings, B is the number of easy calvings misclassified as difficult ones, C is the number of difficult calvings misclassified as easy ones and D is the number of correctly detected easy calvings.Additionally, the probability of true positive and true negative response, also known as accuracy, was calculated: The differences between probabilities were calculated for the training and validation sets together (the same was applied to the estimation of CF parameters) using a two-proportion test for unequal variances.
When comparing the quality of the analysed models, Akaike information criterion (AIC; Akaike 1974), which is a measure describing the goodness of fit, was also determined.The model with the lowest AIC value was selected.The criterion was calculated according to the following formula (Liddle 2007): where c is the number of the observed classes, n k is the number of records assigned to class k by a model, o k is the number of observations in class k, d is the number of model parameters and n is the number of observations.
After preparation and evaluation of the ANN and CF quality on the basis of the training and validation sets, the ability of individual models to detect cows with dystocia was verified on the basis of the test set.For the evaluation of detection performance, the aforementioned probabilities as well as receiver operating characteristic (ROC) curves and area under these curves (AUC) were used.The ROC curves enabled to visually compare the detection performance of the analysed classifiers, whereas the AUC values indicated the discrimination power of the models (Fawcett 2004).
In the last stage of the study, the input variables that had the greatest influence on the determination of calving difficulty class were identified.To do this, the ANN sensitivity analysis was performed.The measure was the ratio coefficient (the higher the value the more significant the variable) and rang, which ordered the variables according to decreasing error.In the case of CF, the contribution of variables was defined using the tolerance coefficient T (Grzesiak et al. 2010).

Results
Based on the lowest RMS error for the validation set, the ANN types presented in Table 2 were selected for further analyses.
In the case of CF, all the explanatory variables except FT_PR% were characterized by the deviation from the normal distribution.The assumption concerning the homogeneity of variance was not fulfiled either (AGE, CI and FCM variables).It should be emphasized that the fulfilment of these assumptions is not required when using ANN.The differences in the quality of the classifiers become visible when analysing the probabilities calculated for the training and validation sets (Table 3).
ANN were characterized by the greater ability to detect cows with difficult calvings compared with CF.However, CF almost perfectly identified individuals without calving difficulties, although the level of these indications for ANN was also high and similar to the sensitivity values.Only the RBF network was characterized by somewhat lower level of correct indications of cows from both categories.Accuracy of indications of cows from both distinguished classes was again greater for CF.
The AIC values calculated on the basis of the training and validation sets were: 117.99 (MLP1), 116.44 (MLP2), 328.54 (RBF) and 409.68 (CF).The prepared ANN and CF were then used for the detection of difficult calvings on the basis of the test set, which comprised information on the new calvings not presented previously either to the ANN or CF.A general trend in the values of the individual probabilities on the training and validation sets remained on the test set (Table 3).ANN were characterized by a high and similar percentage of correct detection of cows with difficult and easy calvings.
Only the RBF network showed lower level of sensitivity and specificity, whereas for CF, the disproportion between the detection of individuals with difficult and easy calvings was observed, to the disadvantage of the former.The highest accuracy was exhibited by CF.
The AUC value indicating the discrimination power of the model (Figure 2), determined on the basis of the test set, was the highest for MLP1 and the lowest for the RBF network.
In the last stage of the study, the variables with the significant contribution to the determination of calving difficulty class were indicated.The sequence of these variables is given in Table 4.

Discussion
As it can be seen from Table 2, the RMS error for the analysed classifiers was low (below 0.3), which testifies to the good quality of the analysed models (StatSoft 2000).The non-fulfilment of the assumptions concerning the normal distribution of variables and homogeneity of variance can be tolerated at greater sample sizes (Stanisz 2007).However, it does not remain without influence on the results of classification by means of CF.When using CF, one should also take into account the collinearity which, in spite of including modified explanatory variables in this study, may always appear.
The accuracy calculated on the basis of the training and validation sets was in general similar to that obtained by other authors.Morrison et al. (1985a, b) applying discriminant analysis to the prediction of dystocia in beef cattle obtained the percentage of correct classifications equal to 82-85 %, whereas Basarab et al. (1993) obtained the values in the range 85-89 %.Montgomery et al. (1987) investigating mastitis detection in cows by means of discriminant analysis obtained the coefficient of correct classification amounting to 0.78.
The applied goodness-of-fit criterion (AIC) allowed us to conclude that the best quality was exhibited by the MLP2 network.
When discussing the individual probabilities defining the quality of detection, it should be first emphasized that the rate of indication of cows with difficult calvings, that is, sensitivity is definitely more important for the breeder than the effectiveness of indicating cows with easy calvings, that is, specificity.The misclassification of a cow with calving difficulty is not as dangerous as indication of cow that will not have such problems, which can »dull« breeder's vigilance.In this respect, CF did not turn out to be useful in practice and the higher probability of indicating an animal with problems can be obtained in a purely random manner.In the studies concerning the application of ANN the sensitivity was, in general, high and similar to that obtained in the present work.In the research on mastitis detection it ranged from 0.24 (Yang et al. 1999) to 0.93 (Cavero et al. 2008).In the study on the oestrus detection in cows (Krieter et al. 2006) sensitivity amounted to 0.78, whereas in the lameness detection in cattle (Pastell & Kujala 2007) 100 % of cows were correctly diagnosed.In the studies on the application of discriminant analysis the level of sensitivity amounted to 0.05-0.32(Basarab et al. 1993), 0.39 (Montgomery et al. 1987) and 0.57 (Morrison et al. 1985b), so it was relatively low and similar to the values for CF in the present study.
Specificity obtained in this research for ANN was lower than that in the studies by Yang et al. (1999) and Yang et al. (2000), where it amounted to 0.83-1.00and 0.67-0.85,respectively.In other studies on mastitis detection these values ranged from 0.38 (Cavero et al. 2008) to 1.00 (Nielen et al. 1995, Wang & Samarasinghe 2005).In the study by Krieter et al. (2006) the percentage of correctly classified cows without oestrus was 100 %, whereas Pastell & Kujala (2007) obtained the percentage of correctly indicated cows without lameness amounting to 58 %.For discriminant analysis, specificity was 0.90 (Morrison et al. 1985b), 0.91 (Montgomery et al. 1987) and 0.73-0.99(Basarab et al. 1993).The specificity obtained in the present study was in general similar to the aforementioned values, which proves certain »carefulness« of the neural classifiers in the indication of classes comprising animals without calving problems, mastitis or lameness, whereas the strictly statistical methods did not show such »carefulness«, which is disadvantageous from the breeder's point of view.
Accuracy given in the literature was in a similar range as that obtained in the present work and was not lower than 50 %.For example, in the study by Yang et al. (1999) these values amounted to 0.62-0.99 and in the work by Yang et al. (2000) they ranged from 0.55 to 0.78.Pastell & Kujala (2007) and Morrison et al. (1985b) obtained accuracy of 0.96 and 0.84 respectively.According to Basarab et al. (1993), 68-93 % of heifers were correctly diagnosed, whereas Montgomery et al. (1987) studying mastitis detection by means of discriminant analysis obtained the maximum accuracy on the test set of 0.75.
Based on the AUC values for the test set, it can be concluded that the greatest discrimination power was exhibited by MLP1.In the study by Yang et al. (1999) the AUC values were somewhat higher and ranged from 0.77 to 0.87, whereas in the study by Pastell & Kujala (2007) this value was 0.86, indicating the high ability of the model to discriminate between healthy and lame cows.
In the last stage of the research, classification variables that had the greatest influence on the determination of calving difficulty class were indicated (Table 4).The most significant variable was SEASON.According to Meijering (1984), in our climatic zone, higher percentage of dystocia cases can be observed in autumn and early winter, which may be caused by less intensive supervision of cows by farmers during the spring-summer season.An easier access to pasture in summer, higher level of physical exercises and longer days are additional factors favouring easy calvings (Zaborski et al. 2009).The next important variable was BCSI, which is a difference between the mean body condition and condition at calving.Both too low and too high body condition at calving has an adverse influence on the course of parturition (the optimum BCS is regarded as 3.25-3.75points on a 5-point BCS scale; Zaborski et al. 2009).Roche et al. (2009) stated that there is an association between the changes of body condition in cows during the periconceptional period and the size of calf and its sex at subsequent calving.Since greater calf size and the male sex are positively correlated with calving difficulty, the BCS change during the early stage of gestation may have connection with the difficulty of the next calving.Also, too high condition during the dry period prolongs gestation, increases the foetus weight and favours difficult calvings (Czerniawska-Piątkowska et al. 2005).Moreover, cows of a higher BCS at calving have more intrapelvic fat reserves and thus decreased pelvic area, which may be associated with dystocia (Avendaño-Reyes et al. 2010).
Also, the CYP19-PvuII genotype (CYP19 variable) played an important part in the determination of the course of calving.The CYP19 gene codes for an enzyme (aromatase cytochrome P450), which catalyses a key step in oestrogens biosynthesis.Their increased level during the antepartum period has a significant influence on the late gestation and calving (Fürbass et al. 2010).In ruminants, a decreased level of oestrogens and their reduced secretion just before parturition are associated with an increased risk of abortion, retained placenta and dystocia (Fürbass et al. 2010).In cows with difficult calvings, caused by incomplete dilatation of the cervix, the level of oestrogens was significantly lower than that in the individuals without calving difficulties (Janowski & Zduńczyk 1995).However, the effect of the two remaining genotypes (ERα-BglI and ERα-SnaBI) was smaller.The PREG variable had a high value of ratio coefficient either.Meijering (1984) showed that gestation length was phenotypically correlated with difficult calvings and that their more frequent occurrence was associated with longer gestation.Also, shorter than normal gestation may cause dystocia as stated by Niskanen & Juga (1997), who justified this by indicating that calves are then smaller and weaker.
The %HF was the next variable of high significance.In general, the improvement of the Blackand-White cattle using the semen of Holstein bulls, conducted in Poland resulted in the increased occurrence of difficult calvings compared with the individuals of the pure Blackand-White breed (Trela et al. 1996).Also, in the world, the increasing proportion of genes of the Holstein breed originating in North America in the genotypes of Friesian cows in Sweden, Denmark, Great Britain, Israel, Ireland, and of Jersey cows in Australia affects calving difficulty resulting in greater birth weight and longer gestation (Mee 2008).For perceptrons, the PEAK and AGE variables were also quite important.However, it is difficult to explain connection between the first of these two variables and dystocia, since most studies on this subject have not shown such relationship (Erb et al. 1981, 1985, Martin et al. 1982, Curtis et al. 1985, Erb 1987).An effect of the two other variables associated with milk yield (FIRST_2 and LAST_2) was smaller, whereas in the case of age at calving, it is known that younger cows usually have more difficult calvings irrespective of the number of parturitions and that the age of cows is directly related to their body size and indirectly to the size of their calves (Zaborski et al. 2009).The sequence of remaining variables varied depending on the classifier used.
The tolerance coefficient values and the significance of F statistics in the case of CF confirmed, in general, the results of ANN sensitivity analysis; although one more variable related to yield (FT_PR%) appeared to be significant for CF.
It should be emphasized that all the diagnostic variables except for genotypes were obtained from the data bases of herd management software.Only acquiring information on the genotypes can be problematic in the breeding practice at present.However, taking into account a relatively low cost of genotyping per one individual in relation to the considerable financial losses associated with dystocia, it can be expected that such analyses will become more popular in the near future.According to the results obtained in the present study, the determination of CYP19-PvuII genotype is sufficient because the effect of the two remaining genotypes on dystocia detection was smaller.
Based on the performed research, it can be concluded that the individual types of ANN used for the classification of calving difficulty in dairy cows were characterized by high quality.Also, the detection of individuals with dystocia performed by means of ANN was effective, whereas the usefulness of CF in this respect was dubious.The variables with the greatest contribution to the determination of the calving difficulty class were: SEASON, PREG, CYP19, BCSI, %HF, PEAK and AGE.Due to the significance of the BCSI variable, it can be suggested that breeders assess body condition in order to obtain additional indicator in dystocia detection in cattle using neural classifier.
Figure 1The division of calving records into subsets

Figure 2
Figure 2 Receiver operating characteristic (ROC) curves and area under curve (AUC) values for the detection of difficult calvings

Table 1
Mean values of the analyzed variables (standard deviations in parentheses)

Table 2
The structure of artificial neural networks (ANN) and the root mean square (RMS) error for ANN and classification functions (training and validation sets) A, B, C, D, E P≤0.01