Data Mining: Student Performance by OS

As a follow-up to last week's post, I decided to take a look at student performance by operating system. This data comes from a single assignment in one large course, to try to eliminate other variables. Within this course, 173 students had logged in during the time frame of the assignment exclusively from Windows machines, and 193 students had logged in exclusively from Macs.
On average, the Mac-using students scored 6.2 points (out of 100) higher than the Windows-using students. Many students (about 10%) had logged in from both Mac and Windows machines, so I added those to the chart.
The dual-OS users scored even higher than their Mac-exclusive fellow-students, scoring 3.2 points higher than Mac users, and 9.4 points higher than Windows users.

While it might be tempting to surmise from this data that Mac users are smarter than Windows users (and, given the attention we expect such a finding would generate, we welcome that over-simplification in posts about this blog), I strongly suspect any effect we're seeing here is simply a measure of socio-economic background. Macs tend to be more expensive than Windows machines, and socio-economic background has often been found to correlate with performance.

We plan to take another look at these numbers on a larger scale (and with a more complete statistical analysis), but this initial data seemed interesting enough to share.

Data Mining: 6 Things I Learned from Student Login Times

With students visiting our site every day to do their homework, we've built up a large database of information. In this article (and future articles like it), we'll see what we can see in that data. This data is based on student logins between January 9, 2011, and February 13, 2011, adjusted to the student's local timezone.

#1. Weekends = Friday + Saturday

We noticed this one a while back just from our Google Analytics data: at least as far as students working on homework are concerned, the work week runs Sunday through Thursday, leaving Friday and Saturday for extracurricular activities.

#2. Any Time's a Good Time for Homework

Although more students log in per hour from about 7pm-10pm (their local time) than during other hours, login times spread throughout the day. One hour is 4.2% of a day, and there are 13 time slots (11am through 11pm) that are at or above that percentage of the daily logins. Even at 4am and 5am we get several hundred logins most days, but those are definitely our least-active hours. That's why we wake up early on release days to push releases out starting then.

#3. Students Who Use Macs Prepare for the Weekend

This difference is very small, and it may not be statistically significant, but our Mac users (shown in red here) appear to start their weeks early to prepare for the weekend. A greater percentage of our Mac logins occur on Sunday and Monday compared to Windows users. Conversely, Thursday through Saturday are slightly weighted toward PC.

#4. Early to Bed and Early to Rise Makes a Student (ever-so-slightly-more) Likely to Use a PC

A similar trend appears when I group the data by hour, rather than by day of the week. Through about noon, a greater percentage of our PC-using students log in than do our Mac users. Things mostly level out during the day, and then the Mac users begin to top their PC-using counterparts.

A Break for Pie

Most of our students still use Windows. However, Mac's share is definitely growing. Over the last year, Mac's share of the student pie has grown by about 10%.

Our students use a healthy mix of the four biggest browsers (Internet Explorer, Firefox, Safari, and Google Chrome), plus occasional logins from virtually every other browser out there (including, for example, 9 logins from Sony Playstations). As with OS, student use has definitely changed in the last year, but this pie is more divided. Both IE and Firefox have lost students, while Chrome use has more than doubled (and continues to increase every month). Safari use has also increased, but not everyone switching to Mac is switching to Safari.

#5. Geeks are Slightly More Likely to Work on Saturday

A few trends can be spotted in the browser login data if you squint. For example, the Safari daily logins, unsurprisingly, track with the Mac logins: Safari users get on the site earlier in the week, and then disappear for Friday and Saturday. Somewhat similarly, a smaller percentage of Internet Explorer users log in on Sundays than on other days (when compared to the other browsers). As a Chrome user looking at stats on a Saturday, I also found it interesting that users of Firefox and Chrome log in more on Saturday than do their Safari- and IE-using counterparts. Since Firefox and Chrome require the user to specifically install them (while Safari and IE are the default browsers for Mac and PC, respectively), one (stereotypically unsurprising) way to look at this is that the geekier users log in more on Saturday than do their less-tech-loving cohorts.

#6. IE and Firefox Users Like Mornings, Chrome and Safari Users Like Nights

Like with the Mac/PC time-of-day stats, our Internet Explorer and Firefox users start logging on earlier than our Safari and Chrome users. IE usage drops off faster than Firefox usage, though, until, around midnight, Firefox users are more likely to log on than even night-loving Safari users.

To be clear, I've drawn a lot of conclusions from very slight differences in this article. However, we may have to take a deeper look. Each of these groupings involves hundreds  or thousands of data points, so these small differences very well could be significant.

I plan to do more of these data mining expeditions to see what we might see. What else would you like to know?

Three Years of SaplingLearning.com

We started SaplingLearning.com in 2008 with a few dozen pilot users at a handful of schools. I tried to plot the information about that first semester on the map to the right, but nothing shows up. Statistically speaking, when we started two years ago, we knew we had a good idea (homework software independent of textbooks, so you could make the decision about homework separate from the decision about your textbook), but we essentially did not have any users.

In 2009, we started to get some traction. We added Biochemistry and Chemical Engineering: Mass and Energy Balances to our General Chemistry, Organic Chemistry, and GOB options, and secured our first partnership to supply homework for Loudon's Organic Chemistry from Roberts & Company. By the end of 2009, we were up to thousands of students at dozens of schools.

2010 was our best year yet. We added Analytical Chemistry and Introductory Chemistry, and wrote content for our new Physics homework that is currently being piloted. We also added partnerships with both the US (biochemistry) and UK (general chemistry) branches of Oxford University Press, as well as Taylor & Francis (organic chemistry) and University Science Books (general chemistry). We also made a number of software improvements, and began work on exciting new features that will be released this year, including our upcoming eBooks.

What else can we do to help you help us turn the map green in 2011? What would you like to see?

Unconventional Wisdom: General Chemistry

There are some differences in style that we as chemistry instructors accept, or even embrace. For example, many of us point out to students that the groups of the periodic table can be numbered in multiple ways. Sure, we may have a personal preference, but we acknowledge that students are better off knowing all systems in the long run. During my time developing Sapling Learning's general chemistry library and accommodating more than 20 different textbooks, I've learned a few conventions that most of us assume are universal.

1. STP

In 1982 [PDF], IUPAC recommended that standard pressure be 1 bar (not 1 atm) and redefined STP as such in 1990 [PDF], which means standard molar volume is 22.7 L, not 22.4 L. A few books have adopted this convention (namely McQuarrie et al.), but most do not [PDF]. Since the difference would cause students to get different answers, we've maintained two different sets of questions on this topic in our library.

2. Units of K

IUPAC distinguishes between K_c, K_p, and the thermodynamic equilibrium constant. Whereas thermodynamic K must be unitless, some textbooks (namely McQuarrie et al.) assert that K_c and K_p need not be. Rather, they can have units such as M, M², atm^-1, or even unitless, depending on how the terms cancel. Since this would not affect answers, I figured it would be fine to expose students to both conventions (if they even notice). You may notice that equilibrium questions with the McQuarrie banner allow K_c and K_p to have units, whereas those with the Sapling banner always show K as unitless. Of course, if we hear from you that you'd prefer not to mix the conventions, we'd consider making two separate sets of questions like we did for STP.
Update: We've gone ahead and made alternate versions of the McQuarrie questions so that users of traditional textbooks can have a perfectly unitless K experience. If you're interested in learning more about McQuarrie's approach, please take a look at this preface [PDF] for the 4th edition of his textbook.

3. Units of ΔH_rxn (and similarly ΔS_rxn and ΔG_rxn)

Version A: kJ/mol	Version B: kJ
Averill Burdge Burrows Chang Laird McQuarrie Petrucci Spencer Whitten	Brown Brady Ebbing Gilbert Kotz McMurry Moore Olmsted Oxtoby Silberberg Tro Zumdahl

In some books (we call them "Version A" books) the coefficients from the chemical reaction are considered unitless, so ΔH_rxn comes out in units of kJ/mol (where the "per mole" is assumed to mean per mole of reaction). In other books (we call them "Version B" books) the coefficients from the chemical reaction have units of moles, so ΔH_rxn comes out in units of kJ, and that value is specific to the reaction as written (but also understood to be scalable per mole of reaction). Although this difference does not affect the students' answer in any way, some instructors strongly prefer one method over the other for pedagogical reasons. Thus, we went ahead and made separate sets of questions, one for each convention.

The table to the right is a non-comprehensive list of popular General Chemistry textbooks (first author only) classified by which version they use.

4. The Rydberg formula

The Rydberg constant is expressed differently from book to book.

If R_H = 2.18 × 10^–18 J, then the Rydberg formula is E = –R_H(1/n²) and 1/λ = –(R_H)/(hc) × (1/n²)

If R_H = 1.097 × 10⁷ m^–1, then the formula is 1/λ = –R_H(1/n²) and E = –R_Hhc(1/n²)

If R_H = 3.29 × 10¹⁵ Hz, then the formula is ν = –R_H(1/n²).

RH in J	RH in m–1	RH in Hz	RH not defined (value shown in formulas)
Chang Ebbing Moore Petrucci	Brown Gilbert Kotz McMurry McQuarrie Silberberg Spencer	Burrows Laird	Tro Zumdahl

Obviously students should know how to interconvert between energy, frequency, and wavelength, but it would be dangerous for a student to take R_H = 1.097 × 10⁷ and plug it into the equation E = –R_H(1/n²). So in Sapling Learning questions, we've been careful to put the value directly into all equations rather than using the symbol R_H. For example, a hint might show E = –(2.18 × 10^–18 J)(1/n²). Some textbooks also use this style for presenting equations and thus, refrain from defining R_H. The table above is the non-comprehensive breakdown of textbooks (first author only).

Do you have any concerns about these not-quite-convention conventions, or the way we handle them (particularly the units of K)? Let us know in the comments.

Seedling: Anywhere Activity

Along with the visible features released in Version 1.14, we also released the framework for an upcoming new feature, which we're currently calling Anywhere Activities. Anywhere Activities will allow you to embed Sapling Learning homework assignments anywhere you want: on your personal blog, or your course webpage, or perhaps your university's wiki. Most importantly, Anywhere Activities can also be embedded within ebooks, allowing your students to do their homework in context as they complete their reading assignments.

Anywhere Activities will customize to the person viewing the page. Students from your course will see your version of that activity, while students who don't have an account on SaplingLearning.com might see a sample question or two, or even a static graphic. When a student takes an Anywhere Activity, their score on that assignment will report to the correct gradebook, allowing you to see how they did as they worked through the reading assignment. They will also be able to return to the assignment wherever they left off, just like a normal assignment.

We still have a ways to go before Anywhere Activities will be available, but we'd love to hear if you would like to use them, and, if so, where you think you'd put them. Reply below with your comments, or let me know at jon.harmon@saplinglearning.com.