Show Your Work: CBE Return on Investment – Results and Explainer

Several years ago, a trade journal arrived in my office with a cover story declaring that calculating Return on Investment (ROI) was like “Nailing Jell-O to the Wall”. I found this statement to be a bit misleading. First, ROI is really not that complicated, if you’re willing to dig for the data and go through some calculations. Second, everyone knows that hanging Jell-O is a bit passé. Instead, it should be displayed in faux-distressed cabinetry alongside old-tymey apothecary bottles.

So, when Capture Higher Ed’s chief operating officer, Matt Cobb, asked me to calculate the ROI for Capture Behavioral Engagement (CBE), I jumped right in and found that CBE generated an excellent return of 25X for partner institutions.

This means that for every dollar spent on CBE, an institution, on average, can expect to yield $25 in net tuition revenue from Fall 2017 students influenced by the technology.

Then, he said the words that have struck terror into my heart since fourth-grade math: “Show your work.”

Alright Matt. Your wish is my command …

Defining Return on Investment

For this analysis, ROI was calculated using the following formula

ROI =Show Your Work

 

Translation: How many times would the cost of CBE be returned in net tuition from CBE-influenced enrollments over a six-year period?

To get at this figure, we will need to break the equation into constituent parts, which have their own constituent parts. So buckle up.

Estimating Six-Year Net Tuition Revenue

There is a lot of math here, but the premise should be well understood by enrollment managers. Your FTFT class starts out at a certain size and composition in Year 1. Each year, through attrition and graduations, that number diminishes until the end of the six-year window recorded by IPEDS.

By estimating the size of that class for each year, one can approximate gross tuition. Then, by subtracting out the total institutional aid one arrives at the net tuition revenue over that full six-year period.

Finally, by dividing that total net tuition revenue amount by the FTFT cohort, you land at an estimate for the total net tuition revenue expected per FTFT over a six-year period.

Step #1 Calculate Diminishing Size of Cohort Over Six Years

Yr1 = First-time, full-time cohort size in 2015
Yr2 = (Yr1 X Yr1Retention)
Yr3 = Yr2 – (Yr2 X .90)
Yr4 = Yr3 – (Yr3 X .95)
Yr5 = Yr4 – (Yr4 X 4YrGradRate)
Yr6 = Yr5 – (Yr5 X 5YrGradRate

Step #2 Calculate Gross Tuition Per Year

 GrossTuitionYr1 =

(PctResident X Yr1) X ResTuition +
(PctNonRes X Yr1) X NonResTuition

 GrossTuitionYr2…Yr[i] =

(PctResident X Yr2…[i]) X ResTuition +
(PctNonRes X Yr2…[i]) X NonResTuition

Step #3 Calculate Net Tuition Per Year

NetTuitionYr1 = GrossTuitionYr1 – InsGrantsFTFT
NetTuitionYr2…Yr[i] = GrossTuitionYr2 – InsGrantsFTFT

Step #4 Calculate 6-Year Lifetime Tuition per FTFT

Show Your Work

Estimated Incremental Enrollments Influenced by CBE

Here we are trying to estimate how many additional enrollments an institution can expect based on students who were influenced by four CBE superpowers: identification, dynamic content served, dynamic content clicked and organic progressive id forms.

Step #1 Query Baseline Data Set for Fall 2017 Prospects

This query established the baseline suspect to enrollment rate for all students in the institution’s prospect pool. Statuses were assigned to reflect a traditional admissions pipeline.

For some institutions without adequate enrollment data, the number of deposits were used as a proxy.

Step #2 Calculate Standard Suspect-to-Enrollment Conversion Rate

Show Your Work

Step #3 Query CBE Data for Identified Students

This query pulled data for students who were identified using Capture Behavioral Engagement, but did not receive dynamic content.

Step #4 Calculate Standard Suspect-to-Enrollment Conversion Rate for CBE Identified Students

Show Your Work

Step #5 Query CBE Data for Students With Served Dynamic Content

This query pulled data for students who were both identified in CBE and who were served dynamic content but did not click.

 Step #6 Calculate Standard Suspect-to-Enrollment Conversion Rate for CBE Dynamic Content Served

Show Your Work

Step #7 Query CBE Data for Students who Clicked Dynamic Content

This query pulled data for individuals who were identified, served dynamic content, and clicked on that content.

Step #8 Calculate Standard Suspect-to-Enrollment Conversion Rate for CBE Dynamic Content Clicked

Show Your Work

Step #9 Query CBE Data for Students who Filled Out Organic Forms

This query pulled data for individuals who were introduced into the CBE data stream by filling out an organic progressive ID form but were not otherwise found in institutional data tables.

Step #10 Calculate Standard Suspect-to-Enrollment Conversion Rate for Students who Filled Out Organic Forms

Since organic forms were a new CBE super power for the 2017 class, and since statuses were not always a ”suspect” it was unclear which conversion rate should be utilized for extrapolation. To err on the conservative side, we opted for the rate for students who clicked on dynamic content, even though one could argue that filling out a form is a more powerful signal than clicking on dynamic content.

Show Your Work

Step #11 Estimate Incremental Enrollments from CBE Identification

This is where the fun begins. Essentially, we calculated the difference in various conversion rates and then multiplied by the total number of prospects. This produces an estimate of the raw number of incremental enrollments influenced by a particular CBE super power. Critically, the first three are nested, meaning that the difference in the conversion rate excludes the students already attributed to the earlier calculations. In other words, each of these equations are incremental students on top of what has already been counted.

Show Your Work

Step #12 Estimate Incremental Enrollments from CBE Dynamic Content Served

This is essentially the same as Step #10, but calculating the value for the number of CBE-influenced enrollments for students who were identified and served dynamic content.

Show Your Work

Step #13 Estimate Incremental Enrollments from CBE Dynamic Content Clicked

Step #12 was repeated to calculate incremental, CBE-influenced enrollments for students who were identified, were served dynamic content, and clicked on that content.

Show Your Work

Step #14 Estimate Incremental Enrollments from CBE Organic Forms

Step #13 was repeated to calculate incremental, CBE-influenced enrollments for students who filled out organic forms. The rate difference mirrors that of students who clicked dynamic content because of the assumptions made in Step #10.

Show Your Work

Step #15 Remove Nested Duplicates

We now have four distinct values for the incremental CBE-influenced enrollments:Show Your Work

Although our original queries and methodology should have eliminated double-counting we wanted to be completely sure that the individual numbers were exclusive. Therefore, we subtracted the potentially overlapping students from the parent group (with the exception of organic forms which are a unique group already).

Show Your Work

Show Your Work

Show Your Work

Show Your Work

Step #16 Attribute Credit Where It’s Due

At Capture, we believe that the enrollment of a student involves a complex set of factors, behaviors and interventions. As such, we find it patently ridiculous that any single intervention should get full “credit” for a particular enrollment. To ameliorate this problem, we are working on more robust attribution modeling to assign the appropriate weight to any given intervention. Until we have those data fully deployed, we have made some rough estimates which effectively “penalize” the Capture-influenced numbers, in order to assign proper credit to the school’s brand, the admissions recruitment efforts, and anyone or anything else involved. In our current estimates, we assigned 5 percent credit for identification, 10 percent to served dynamic content, 25 percent to clicked dynamic content and 35 percent to organic forms, which are students that were otherwise invisible to the institution.

Show Your Work

Admittedly, these numbers can be viewed as arbitrary, but we view it as a critical fulfillment of one of Capture’s core values to be humble as well as a step forward from the “I touched it, I’m taking credit” mentality that is prevalent in the industry.

Tie It All Together

At this point, the heavy lifting is done and the rest should seem pretty straight-forward.

Step #1: Add Up the Incremental Enrollments

By adding the four different CBE super powers together, we estimated a total number of incremental, CBE-influenced enrollments for the Fall 2017 class.

Show Your Work

Step #2: Multiply by the estimated 6-year, per student, net tuition

Show Your Work

Step #3: Subtract the Cost of Capture Behavioral Engagement

Show Your Work

Step #4: Divide by the Cost of Capture Behavioral Engagement

Show Your Work

Step #5: Report the Results and Discuss

 

Results and Discussion

We found, that among the 33 partners analyzed for this study, the average return on the investment for CBE was 25X. This is, by any measure, a worthwhile return, but there are several reasons why we believe that this number is potentially underestimating the true value.

There are a number of assumptions included in this analysis that are intended to be conservative. For instance, we held a lot constant in the IPEDS data, particularly the cost of tuition and fees. Although the specific tuition and aid balances at institutions will vary considerably, it is likely that tuition and fees will increase over the subsequent six years, causing a presumed increase in net tuition and subsequent returns on the CBE investment.

Conclusion

CBE works very well at influencing students to enroll at institutions that utilize the technology. After looking across institutions and analyzing the data from 2017, we are confident that the cost of CBE will be returned to the partner many, many times over.

By Brad Weiner, Director of Data Science, Capture Higher Ed