Fuzzy Unordered Rule Induction Algorithm Application in Basic Programming Language Competence: A Rule-Based Model

Objective: A rule-based model using Fuzzy Unordered Rule Induction Algorithm (FURIA) aimed to be developed in this study. Methods/Statistical Analysis: This study utilized a Machine-Learning algorithm called FURIA in developing the model. Graduates of the K to 12 programs in the Philippines who enrolled in the Bachelor of Science in Information Technology in a State University were the respondents of this study. Respondents were assessed using a self-made questionnaire in basic programming using Java programming language and validated both face validation and the internal reliability test with a Cronbach alpha value of 83. Furthermore, the steps performed in this study were the variable definition and data collection, data processing and cleaning, machine-learning algorithm application, verifying and interpreting the model, and reporting the model and synthesizing. Findings: This study revealed seven (7) rule-based models with a substantial rating in Kappa statistics and a very high accuracy rating in predicting the competencies by class. Applications/ Improvements: The researcher believed the need to unveil the competence of the Senior High School graduates in the K to 12 programs of the Department of Education in the Philippines in basic programming, which aimed to retool the type of teaching strategies to fit the competencies of the student.


Introduction
The basic principle in the logic formulation for developing skills in programming is an indispensable and integral function in developing analytical skills for complex problem-solving. Arguments relative to the best approach in teaching programming like direct exposure or taking up with basic programming 1 are very evident particularly when the school adopted the Outcomes-Based Education (OBE). Mathematics is not the only prerequisite of programming subjects, but also an adequate command in English is essential 2 . Anent to this, many literatures claim some cognitive issues on learning computer programmings such as the definition of computer programming, levels of programming skill development, the demands of learning to program, and programming as a cognitive activity 3 . This gap showed a significant impact on the performance of the student in ICT and in the programming course itself.
On the contrary, predicting student performance has been in the limelight for many years in understanding and unveiling the root of how we can help improve the performance of the students and the graduates itself. A study in predicting self-efficacy in test preparation where gender, value, anxiety, test performance, and strategies were conducted utilizing hierarchical multiple regression where the ability in predicting the independent variables in the first model showed R 2 = 0.3, second model was R 2 = 0.211, and the third model was R 2 = 0.108 4 . Another study utilized the same approach in predicting the quality of life after the graduation of senior college students with disabilities which were the stepwise multiple regression analyses 5 . In this research, R-squared where used to determine how close the data are to the fitted regression line, where R-squared is always between 0 and 100%, however, to determine the goodness of fit, it is also essential to look into the residual plots 6 .
In the Philippines, it is very evident that the effect of educational reform made and mandated by the government was the education arm joined together in implementing the Philippine Qualifications Framework (PQF). PQF describes the levels of educational qualifications and sets the standards for qualification outcomes as Senator Villanueva authored and sponsored RA 10968 7 . The Philippine qualification framework 8 assured quality graduates that can add the human resource of the country. The involved agencies in implementing this framework were the Department of Education (DepEd) for primary education, Technical Education and Skills Development (TESDA), and the Commission on Higher Education (CHED) for Baccalaureate to Post-Doctoral programs. Recently, Senior High School Graduates (SHSG) in the Philippines were accepted to enroll regardless of their tracks or specialization in their senior high school on the degree of their choice since it was not stipulated in the recent CHED Memorandum Order 34, series of 2016. In line with this, competencies relative to the field of specialization they have chosen remain unknown since they can choose any bachelor's degree regardless of their tracks in Senior High School.
Based on the different kinds of literature mentioned in the previous discussions, it is very evident that prediction studies merely focused on varied topics and subjects that were not in line with the current study.
In general, predicting SHS graduate competencies in basic programming remain unexplored. Therefore, it is empirical to investigate this emerging phenomenon in order to design a model based on Fuzzy Unordered Rule Induction Algorithm (FURIA).

The Rule-Based Algorithm
The rule-based machine learning algorithm is widely used in computer science where the goal is to compete with other machine learning procedures that identify, learns, or evolves 'rules' to store, manipulate or apply 9,10 . This type of algorithm includes artificial immune systems, association rule learning, and learning classifier systems and other algorithms that rely on the set of rules where each is covering conceptual and contextual knowledge. Rules typically take in the form of {IF:THEN} expression for example {IF <condition> THEN <result>}.
Many researchers had made significant innovation of the rule-based algorithm such as a trainable rule-based algorithm for word segmentation. The algorithm was useful both as a high-accuracy stand-alone segmenter and as a postprocessor that improves the output of existing word segmentation algorithms 11 . Another study utilized the rule-based algorithm for assigning myocardial fiber orientation to computational heart models. The developed model is called the Laplace-Dirichlet Rule-Based (LDRB) algorithm for assigning myocardial fiber orientation to computational heart models with speed, accuracy, and high usability 12 . Recently, an evaluation study between rule-based algorithm and feature selection approaches for classification was made. In the study, it utilized the rule-based algorithm called Prism, which works with the concept of target class and is capable of selecting attributes based on their importance to a particular class 13 .

The RIPPER Algorithm
The RIPPER is a rule-based learning algorithm for classification problems because of its simplicity and compact 14,15 . Also, RIPPER was able to successfully deal with overfitting 16,17 and is an excellent choice for learning a fuzzy system where each rule can lessen the impact of a significant problem in fuzzy systems known as the curse of dimensionality 17 .RIPPER algorithm was developed for classification problems togeneratea crisp rule 15 , to overcome this problem, continuous variable needs to be discretized,and the type of discretization can affect the quality of the rules generated by the RIPPER 18,19 .
Several studies were conducted utilizing the RIPPER algorithm. Repeated Incremental Pruning to Produce Error Reduction or RIPPER is a proportional rule learner; it extracts the rules directly from the data where it works in four stages such as growth, pruning, optimization, and selection 14 . A study on cardiac arrhythmia classification using machine learning 20 showed how the RIPPER perform and was measure through accuracy surpassing other machine learning algorithms. Also, a study in predicting the severity of closed source of NASA defect reports is Vol 12 (12) | March 2019 | www.indjst.org Las Johansen B. Caluza done utilizing RIPPER 21,22 showed high accuracy rating in predicting the target variable.

The FURIA Algorithm
Fuzzy Unordered Rule Induction Algorithm or FURIA is an extended version of a RIPPER developed by 17 . FURIA learns fuzzy rules instead of conventional rules and unordered rulesets instead of rule lists. Fuzzy rules are more general than conventional rules and have some advantages while, most conventional rule learners is a decision list. Below is an example of the antecedent fuzzification algorithm for a single rule r in Algorithm 1 developed by 17 .
For the fuzzification of a single antecedent ( ) it is important to consider only the relevant training data i T D , i.e., to ignore those instances that are excluded by any other antecedent ( ) , We partition i T D into the subset of positive instances, i T D + , and negative instances, i T D − . To measure the quality of a fuzzification, the rule purity will be used:

Model Development Methodology
The study utilized Machine Learning (ML) process to derive a model and used to predict the competence of the graduates of Senior High School (SHS) enrolled in a State University in the Philippines in basic programming using Java programming language as a tool. ML algorithms build a mathematical model of sample information, experienced as training data to make predictions or decisions without being explicitly programmed to perform the task 23 . The processes performed in this study are shown.

Variable Definition and Data Collection
The variables used in this study are based on the personal information and perceived competence on the different generic topics in basic programming of the respondents. Table 1 shows the variable definition in this study. The instrument used in this study is a self-made questionnaire and was validated both face validation and reliability test. In the face validation, the research asked an English language expert to check, revised the grammar and its reading understandability with gunning fog index value of 13, and considered a college freshman level 24 while the internal consistency test was conducted after. Here were 30 freshmen student who have the same characteristics of the respondents was utilized to answer the questionnaire and measured its internal consistency with a Cronbach alpha value of 0.83 and considered better or accepted in a Likert Scale questionnaire. In the instrument, the competencieshave specific descriptions describing the different topics such as basic programming, operators, loop, and arrays. Each description, the respondents will assess their knowledge of the different topics.

Graphical Presentation of All Attributes
This study utilized convenience sampling on the basis where the members of the sample are conveniently accessible. The instrument was personally distributed and retrieved by the researcher to the 70 freshmen students enrolled in Bachelor of Science in Information Technology in a Philippine State University.

Data Preprocessing and Cleaning
The datasets collected in this study was encoded using MS Excel and saved with XLSX extension name. Removal of information that was inconsistent and incomplete was performed to maintain its consistency and reliability of datasets. The refined datasets are now converted into Comma Separated Value or CSV format; this is because the version of the R programming software accepts only this kind of format. This process in data mining technique that involves the transformation of raw data into understandable format and removal of incomplete, inconsistent, and lacking in certain behaviors or trends, and is likely to contain errors is called data preprocessing and cleaning 25,26 .

Machine Learning Algorithm Application
This study utilized the Weka application version 3.8.3 in applying the FURIA to 70 datasets. FURIA was used in generating rules in predicting programming competence of senior high school graduates enrolled in Programming 1 using Java Programming language as a basis for developing teaching strategies fit to their level of competence. Fuzzy rules are more general than conventional rules and have some advantages, such as FURIA learns fuzzy rules instead of conventional rules; it makes use of an efficient rule stretching method 17 .

Verifying and Interpreting the Model
10-folds Cross-validation was applied in this study to evaluate and test the generated model. Cross-validation is a technique to evaluate predictive models by partitioning the original sample into a training set to train the model, and test set to evaluate it 27 . In k-fold cross-validation, the original sample is randomly partitioned into k equal size subsamples. Of the k subsamples, a single subsample is retained as the validation data for testing the model, and the remaining k-1 subsamples use as training data. The cross-validation process is then repeated k times (the folds), with each of the k subsamples used exactly once as the validation data. The k results from the folds can then be averaged (or otherwise combined) to produce a single estimation 28 . The benefit of this method is that all observations are used for both training and validation, and each observation is used for validation exactly once 29    The perfect test (Figure 1) will have a ROC curve that passes through the upper left corner(corresponding to 100 % sensitivity and 100 % specificity). Generally, the closer a ROC curve is to the upper left corner, the better the test is 30 .

PRC Area: The precision-recall curve is a Plot of Precision Versus (PPV) recall (sensitivity) for all potential cut-offs for a test.
The perfect test will have a PRC that passes through the upper right corner (corresponding to 100% precision and 100% recall), Figure 2. Generally, the closer a PRC is to the upper right corner; the better the test is 30 . The reliability of the model was measured using the Kappa index (Kappa statistics) as shown in Table 2 31 .

Reporting the Model and Synthesizing
An in-depth analysis of the model showing the importance and the implication towards the real-time application of the generated model is presented in this segment. The analysis of the result is supported by known literature showing how the gaps were answered in this study.

Results and Discussion
The model generated in FURIA shown below consists of seven (7)  Finally, if the perceived competence of the respondents in conditional statements and basic java programming skills is positive +inf but their knowledge in operators with basic java programming skills is negative +inf would result to "Newbie" with a CF = 0.77. These skills are required and prerequisites to more advanced and complexproblem-solving in the real world situation. In the implementation of the K to 12 programs in the Philippines, the Department of Education enforces the different schools including private schools to operationalized its offerings in the Senior High School Programs that to include ICT Track.
The government envisioned to have a more competent human resource in the future through its graduates. This implementation is supported by the different agencies that to include the Technical Education and Skills Development Authority (TESDA), Commission on Higher Education (CHED), Department of Labor and Employment (DOLE), Professional Regulation Commission (PRC), and others. This implementation is through the Republic Act No. 105533, also known as the Enhanced Basic Education Act of 2013 32,33 . However, despite the effort of the national government in the implementation of the K to 12 programs, many still lack in materials and facilities 34 . Hence, the implementation of digital services requires problem-solving, design skills, logical thinking, understanding how computers and networks operate, and most importantly programming competence 35 .    incorrect classification with a kappa statistics of 0.77 as shown in Table 3. The Kappa statistic (or value) is a metric that compares an Observed Accuracy with an Expected Accuracy (random chance). The kappa statistic 36 is used not only to evaluate a single classifier but also to evaluate classifiers amongst themselves. Therefore, the probability of agreeing in predicting the competence in Basic Programming is 77% (good agreement or Substantial) as perceived by the respondents. The model identified three (3) classes such as Expert, Newbie, and NKALL out five (5) classes. As perceived by the respondents as illustrated in Table 4. The TP rate of class Expert and NKALL showed a high prediction rate with 82.1% and 100% respectively, while the TP rate for Newbie also showed a good prediction rate of with 66.7%. This is supported by a low FP rate of the three (3) classes with 6.5% (Expert), 8.9% (NKALL), and 4.7% (Newbie). Meanwhile, in terms of precision of the different classes, it also showed an acceptable rating such as 94.1% (Expert), 86.2% (NKALL), and 57.1 (Newbie). Moreover, the accuracy of the model by classes was found through the ROC curve, where the higher the value, the more beneficial the test solution. In this case, all the classes received a very high accuracy in ROC Area such as 92.5% (Expert), 96.7% (NKALL), and 96.9% (Newbie) resulting to a Weighted Average of ROC Area of 94.4%.The model showed its classification score in confusion matric as shown in Table 5 where: • 32 instances are correctly classified as Expert, and two (2) misclassified instances rated Expert when it is a Newbie. • 25 instances are correctly classified as NKALL, and four (4) misclassified instances rated as NKALL when it is an Expert. • There are four (4) instances that are classified as Newbie, and three (3) misclassified instances rated as Newbie when it is an Expert. The model generated was tested using 10-folds crossvalidation. As suggested by Beleites, Neugebauer, and Bocklitz 37-42 , a small microarray dataset with at least 53 to 208 or any considered as small sample size, re-sampling strategies like bootstrap is repeated or iterated k-fold cross validation are the most appropriate tool to sample test. In this case, a Table 6 show how the model was tested where actual value had its corresponding predicted value and was marked by (+) if an error prediction occurs.

Conclusion
The use of FURIA in developing a model, predicting the competencies of Senior High School (SHS) Graduates in a State University in basic programming using Java computer language was developed. The generated model was tested and showed a substantial result in the rates and very high prediction rate or accuracy rate by class. This revelation would help the teachers in devising their teaching strategies and teaching styles based on the capabilities of the learners.
Further research maybe conducted on the learning and teaching styles of these SHS graduates to align their competencies, type of teaching styles and learning styles. Moreover, this model can be used to develop a system that will determine the competencies of the SHS graduates in basic programming using the Java programming language.