Distribution of Programming Aptitude Among

Non-Graduates, Non-IT Graduates and IT Graduates

by Joe Williams, Psychometrics UK

The following set of charts show the distribution of programming aptitude in Non-Graduates, Non-IT Graduates and IT Graduates. The three different groups were tested at assessment centres by UK employers looking for people with the potential to learn IT skills.

A trainee must learn many new skills in a short period of time, with one of the primary skills being programming. This is often the first skill to be learned in an IT career. A trainee will be taught a programming language such as COBOL, C++ or Java depending on the system requirements of the employing organisation. For trainees without prior programming experience this is an obvious first step, but this is also true for IT Graduates. Many IT Graduates will not have learned the specific programming language being used by the employing organisation. Even IT Graduates with coursework in the specific programming language need on-the-job programmer training. So all trainees, regardless of background, will have to learn new skills.

THE DIFFERENT GROUPS

Non-Graduates

These are job applicants aged 16+ who do not have a degree or equivalent level qualification. They are educated to either GCSE, A level, HND or equivalent level. Some have only just finished their education and some are career changers with a variety of educational backgrounds and work experiences. Most will not have had any formal education or work experience in IT.

Non-IT Graduates

These are job applicants who have a degree or equivalent level qualification in a discipline other than IT.

IT Graduates

These are job applicants who have a degree or equivalent level qualification in an IT discipline.

THE TESTS

The tests are designed by Psychometrics, a company that has been designing IT specific skills and aptitude tests for over 30 years. They design tests to high professional standards focusing on job relevance and predictive validity. The two aptitude tests used in the assessment centres were as follows:

Berger Aptitude for Programming Test (B-APT)

A job-related tutorial test, the B-APT requires the candidate to learn a simple, hypothetical programming language and use it to solve a series of problems. The examinees must apply the rules of coding, looping, incrementing and branching as they write short programs in the test booklet. They must also be able to demonstrate “adaptive flexibility” by applying the language rules to progressively more complex problem requirements.

Berger Aptitude for Programming Test – Advanced Form (B-APT AF)

A job-related tutorial test, the B-APT AF requires the examinee to learn a programming language resembling today's newer, more powerful languages. To answer the test questions, the candidate must analyse program specifications; write coded instructions to a hypothetical computer; and apply the language rules to increasingly complex practical situations.

The B-APT is designed for people without prior programming experience and the B-APT AF is designed for people with a minimum of one month or more. The Non-Graduates and Non-IT Graduates groups took the B-APT and the IT Graduates took the B-APT AF. The two tests are respectively measuring basic and advanced level programming aptitude.

THE CHARTS

Each group has a bar chart and a pie chart. The bar charts show the number of examinees on the Y axis against the test score on the X axis. The BAPT test is scored out of 30 and the B-APT AF is scored out of 25. This explains the difference between the X axis numbers for the IT Graduates as they took the B-APT AF.

The pie charts show the test scores converted into four different levels of aptitude; High, Medium, Low, and Little or No aptitude. The comments under the band explain which test scores convert to which band.

For both charts the High aptitude band is coloured orange. In most cases, employing organisations are looking to train people with a high aptitude regardless of the group they are from. The orange segments help to show the different proportions of high aptitude among the 3 groups.

Non-Graduates – Bar Chart

This is a typical distribution for the B-APT test and illustrates the bi-modal distribution of programming aptitude in the general population. There are two groups, those with aptitude and those without. The group without aptitude is peaking at 9 and the group with aptitude is peaking at 27.

B-APT Score Distribution
(Non Graduates)
n = 966

Non-Graduates – Pie Chart

This shows 26% of the group had high aptitude. An employer would need to test 4 Non-Graduates to find 1 trainee with high programming aptitude.

Non-IT Graduates – Bar Chart

This shows the impact of applying highly specified pre-selection criteria before testing. The bi-modal distribution is significantly less pronounced. This shows that Non-IT Graduates have a greater probability of having aptitude than Non-Graduates. However it also shows that almost half of the Graduates did not have a high programming aptitude which indicates that a good degree does not predict programming aptitude on it’s own.

Non-IT Graduates - Pie Chart

This shows 53% of the group have a high aptitude which is more than double that of the Non-Graduates. It’s interesting to note that the medium and low aptitude bands are similar between the Non-Graduate and Non-IT Graduate groups. The change in distribution is due almost entirely to the difference in the No aptitude and High aptitude bands This shows the effect of applying pre-selection criteria such as good degree result.

IT Graduates – Bar Chart

This shows a similar distribution to the Non-IT Graduates. There is still a slight bi-modal distribution but it is again less pronounced. It’s important to remember the two groups sat different tests. The B-APT AF is the Advanced Form of the B-APT and it is a more demanding test. However, it’s also important to remember that this test was taken by IT Graduates. They were also pre-selected, not only on the basis of a good degree result but also on the IT specific content of their course. With the Non-IT group, employing organisations would expect a good proportion to not have aptitude. With the IT group most employers would expect most to have aptitude. The assumption is that anyone with a good IT degree result should have a strong aptitude for programming. This isn’t the case.

IT Graduates - Pie Chart

This shows 58% have a high advanced level aptitude. As mentioned, many employing organisations would expect this to be higher. The real surprise is 23% having little or no advanced level aptitude. This shows that a good IT degree correlates with aptitude but not strongly enough for it to be a useful predictor on its own. This distinction becomes crucial to employers hiring trainees to learn the newer, advanced programming languages. If an employer had hired all of the IT Graduates in this study, it is likely that more than 40% would have failed training. A very costly outcome given today’s training costs.

Conclusions

These results are looking at 3 different groups and 2 different levels of aptitude. Employing organisations usually have a variety of IT jobs which require different levels of aptitude. Some roles will require an advanced level aptitude and others a basic level. Many organisations will only consider IT Graduates for IT job roles regardless of the aptitude level required. It’s easy to understand why. Companies develop recruiting strategies to ensure they hire people who are capable of doing the job being offered. The assumption is that IT Graduates have proven this capability or aptitude whereas Non-IT Graduates have not. There is nothing in the work experience or education of the Non-IT job applicant to indicate whether they will or will not be capable of learning the job. This makes them a higher risk category than IT Graduates. Many employers assume IT Graduates provide a head start on supplying the business with people ready to do the job.

The test results challenge these assumptions. Firstly, all IT Graduates do not have high advanced level aptitude, in fact 42% do not. Surprisingly, 1 in 4 have little or no aptitude. Secondly, the same degree does not automatically mean the same level of aptitude. This is probably due to the wide diversity of educational institutions and course contents. There is also a difference between succeeding in an educational environment compared to succeeding as on-the-job programmer or IT professional. Employing organisations need to be aware of this when pre-selecting IT Graduates.

Secondly, Non-Graduates and Non-IT Graduates could be considered for jobs which require a basic level of aptitude. The B-APT results clearly show that 1 in 4 Non-Graduates and more than half of the Non-IT Graduates could be successfully trained into programming job roles. Of this group many could well prove to have advanced level aptitude which would be revealed if tested with the B-APT AF after initial training.

With a careful assessment procedure it is possible for employing organisations to broaden their recruitment strategy and select people for different IT job roles from more diverse backgrounds in terms of education and work experience.