Several years ago, a trade journal arrived in my office with a cover story declaring that calculating Return on Investment (ROI) was like “Nailing Jell-O to the Wall”. I found this statement to be a bit misleading. First, ROI is really not that complicated, if you’re willing to dig for the data and go through some calculations. Second, everyone knows that hanging Jell-O is a bit passé. Instead, it should be displayed in faux-distressed cabinetry alongside old-tymey apothecary bottles.
So, when Capture Higher Ed’s chief operating officer, Matt Cobb, asked me to calculate the ROI for Capture Behavioral Engagement (CBE), I jumped right in and found that CBE generated an excellent return of 25X for partner institutions.
This means that for every dollar spent on CBE, an institution, on average, can expect to yield $25 in net tuition revenue from Fall 2017 students influenced by the technology.
Then, he said the words that have struck terror into my heart since fourth-grade math: “Show your work.”
Alright Matt. Your wish is my command …
Defining Return on Investment
For this analysis, ROI was calculated using the following formula
ROI =
Translation: How many times would the cost of CBE be returned in net tuition from CBE-influenced enrollments over a six-year period?
To get at this figure, we will need to break the equation into constituent parts, which have their own constituent parts. So buckle up.
Estimating Six-Year Net Tuition Revenue
There is a lot of math here, but the premise should be well understood by enrollment managers. Your FTFT class starts out at a certain size and composition in Year 1. Each year, through attrition and graduations, that number diminishes until the end of the six-year window recorded by IPEDS.
By estimating the size of that class for each year, one can approximate gross tuition. Then, by subtracting out the total institutional aid one arrives at the net tuition revenue over that full six-year period.
Finally, by dividing that total net tuition revenue amount by the FTFT cohort, you land at an estimate for the total net tuition revenue expected per FTFT over a six-year period.
Step #1 Calculate Diminishing Size of Cohort Over Six Years
Yr1 = First-time, full-time cohort size in 2015
Yr2 = (Yr1 X Yr1Retention)
Yr3 = Yr2 – (Yr2 X .90)
Yr4 = Yr3 – (Yr3 X .95)
Yr5 = Yr4 – (Yr4 X 4YrGradRate)
Yr6 = Yr5 – (Yr5 X 5YrGradRate
Step #2 Calculate Gross Tuition Per Year
GrossTuitionYr1 =
(PctResident X Yr1) X ResTuition +
(PctNonRes X Yr1) X NonResTuition
GrossTuitionYr2…Yr[i] =
(PctResident X Yr2…[i]) X ResTuition +
(PctNonRes X Yr2…[i]) X NonResTuition
Step #3 Calculate Net Tuition Per Year
NetTuitionYr1 = GrossTuitionYr1 – InsGrantsFTFT
NetTuitionYr2…Yr[i] = GrossTuitionYr2 – InsGrantsFTFT
Step #4 Calculate 6-Year Lifetime Tuition per FTFT
Estimated Incremental Enrollments Influenced by CBE
Here we are trying to estimate how many additional enrollments an institution can expect based on students who were influenced by four CBE superpowers: identification, dynamic content served, dynamic content clicked and organic progressive id forms.
Step #1 Query Baseline Data Set for Fall 2017 Prospects
This query established the baseline suspect to enrollment rate for all students in the institution’s prospect pool. Statuses were assigned to reflect a traditional admissions pipeline.
For some institutions without adequate enrollment data, the number of deposits were used as a proxy.
Step #2 Calculate Standard Suspect-to-Enrollment Conversion Rate
Step #3 Query CBE Data for Identified Students
This query pulled data for students who were identified using Capture Behavioral Engagement, but did not receive dynamic content.
Step #4 Calculate Standard Suspect-to-Enrollment Conversion Rate for CBE Identified Students
Step #5 Query CBE Data for Students With Served Dynamic Content
This query pulled data for students who were both identified in CBE and who were served dynamic content but did not click.
Step #6 Calculate Standard Suspect-to-Enrollment Conversion Rate for CBE Dynamic Content Served
Step #7 Query CBE Data for Students who Clicked Dynamic Content
This query pulled data for individuals who were identified, served dynamic content, and clicked on that content.
Step #8 Calculate Standard Suspect-to-Enrollment Conversion Rate for CBE Dynamic Content Clicked
Step #9 Query CBE Data for Students who Filled Out Organic Forms
This query pulled data for individuals who were introduced into the CBE data stream by filling out an organic progressive ID form but were not otherwise found in institutional data tables.
Step #10 Calculate Standard Suspect-to-Enrollment Conversion Rate for Students who Filled Out Organic Forms
Since organic forms were a new CBE super power for the 2017 class, and since statuses were not always a ”suspect” it was unclear which conversion rate should be utilized for extrapolation. To err on the conservative side, we opted for the rate for students who clicked on dynamic content, even though one could argue that filling out a form is a more powerful signal than clicking on dynamic content.
Step #11 Estimate Incremental Enrollments from CBE Identification
This is where the fun begins. Essentially, we calculated the difference in various conversion rates and then multiplied by the total number of prospects. This produces an estimate of the raw number of incremental enrollments influenced by a particular CBE super power. Critically, the first three are nested, meaning that the difference in the conversion rate excludes the students already attributed to the earlier calculations. In other words, each of these equations are incremental students on top of what has already been counted.
Step #12 Estimate Incremental Enrollments from CBE Dynamic Content Served
This is essentially the same as Step #10, but calculating the value for the number of CBE-influenced enrollments for students who were identified and served dynamic content.
Step #13 Estimate Incremental Enrollments from CBE Dynamic Content Clicked
Step #12 was repeated to calculate incremental, CBE-influenced enrollments for students who were identified, were served dynamic content, and clicked on that content.
Step #14 Estimate Incremental Enrollments from CBE Organic Forms
Step #13 was repeated to calculate incremental, CBE-influenced enrollments for students who filled out organic forms. The rate difference mirrors that of students who clicked dynamic content because of the assumptions made in Step #10.
Step #15 Remove Nested Duplicates
We now have four distinct values for the incremental CBE-influenced enrollments:
Although our original queries and methodology should have eliminated double-counting we wanted to be completely sure that the individual numbers were exclusive. Therefore, we subtracted the potentially overlapping students from the parent group (with the exception of organic forms which are a unique group already).
Step #16 Attribute Credit Where It’s Due
At Capture, we believe that the enrollment of a student involves a complex set of factors, behaviors and interventions. As such, we find it patently ridiculous that any single intervention should get full “credit” for a particular enrollment. To ameliorate this problem, we are working on more robust attribution modeling to assign the appropriate weight to any given intervention. Until we have those data fully deployed, we have made some rough estimates which effectively “penalize” the Capture-influenced numbers, in order to assign proper credit to the school’s brand, the admissions recruitment efforts, and anyone or anything else involved. In our current estimates, we assigned 5 percent credit for identification, 10 percent to served dynamic content, 25 percent to clicked dynamic content and 35 percent to organic forms, which are students that were otherwise invisible to the institution.
Admittedly, these numbers can be viewed as arbitrary, but we view it as a critical fulfillment of one of Capture’s core values to be humble as well as a step forward from the “I touched it, I’m taking credit” mentality that is prevalent in the industry.
Tie It All Together
At this point, the heavy lifting is done and the rest should seem pretty straight-forward.
Step #1: Add Up the Incremental Enrollments
By adding the four different CBE super powers together, we estimated a total number of incremental, CBE-influenced enrollments for the Fall 2017 class.
Step #2: Multiply by the estimated 6-year, per student, net tuition
Step #3: Subtract the Cost of Capture Behavioral Engagement
Step #4: Divide by the Cost of Capture Behavioral Engagement
Step #5: Report the Results and Discuss
Results and Discussion
We found, that among the 33 partners analyzed for this study, the average return on the investment for CBE was 25X. This is, by any measure, a worthwhile return, but there are several reasons why we believe that this number is potentially underestimating the true value.
There are a number of assumptions included in this analysis that are intended to be conservative. For instance, we held a lot constant in the IPEDS data, particularly the cost of tuition and fees. Although the specific tuition and aid balances at institutions will vary considerably, it is likely that tuition and fees will increase over the subsequent six years, causing a presumed increase in net tuition and subsequent returns on the CBE investment.
Conclusion
CBE works very well at influencing students to enroll at institutions that utilize the technology. After looking across institutions and analyzing the data from 2017, we are confident that the cost of CBE will be returned to the partner many, many times over.
By Brad Weiner, Director of Data Science, Capture Higher Ed
