The post Progress toward Data Tool and Analytics Maturity appeared first on United InfoLytics.

To help you understand why the stakes of this analytical mistake are high, imagine you teach at a school that offers a bonus for any teacher whose students improve (on average) by 10 percentiles from their prior year state tests. While this might seem a fair measure, it’s fundamentally flawed such that some teachers will have a much harder time attaining this goal, and it turns out that taking averages of percentiles is mathematically illegal. The good news is that there’s an easy fix: use Normal Curve Equivalents (NCEs) instead of percentile rank, and there’s an easy excel formula to do your conversion. We hope you’ll find this video helpful and it’ll help you do the right things with your data! In a hurry? You’re also welcome to watch the abbreviated version here.

Watched the video? Read Nate and Peter’s summary / remix below to ensure you’re ready to apply it to your own data.

Education is a consequential field that ends up generating a lot of data in any given academic year, but understanding education data can be complex. School and district leaders want to measure success, and they want to know what to do after they analyze the data. It’s common for them to want to know things like which classrooms are seeing more or less success in growing students. Since standardized tests might change year to year and people don’t always have good access to rigorous growth norms, one of the easiest ways to see if students are growing is to compare their current percentile rank to an earlier one (typically at the end of the prior school year). For example, a student who ranked at the 45th percentile last year (performed better than 45% of all students taking the test) but is at the 61st percentile rank this year has grown a lot: not only have they kept pace with their peers and grown the typical amount this year, they have made more than a typical year of growth and have passed some of their peers in the process. Even if the tests themselves were of different difficulty levels or offered by different testing vendors we can identify their progress year over year just by looking at the change in percentile rank.

Because comparing percentile ranks is easy and these ranks are included in many datasets, it’s not uncommon to see someone subtract the current year’s percentile rank from the previous year’s to assess how much students grew or regressed relative to the “typical growth” of peers starting at the same incoming achievement. They may also take averages of those percentile ranks before subtracting, producing something like this:

Based on the chart above, a principal could come to the conclusion that teachers A and C are achieving phenomenal growth, and teachers B and D should perhaps be mentored by them to learn what techniques and approaches contribute to such great academic achievement.

However, this analysis is based on a faulty model. This is because percentile ranks are not created equal. Instead, they are distributed unevenly based on the achievement of the students in the sample set. For most valid and age-appropriate tests, student achievement will resemble a bell curve or normal distribution, like this:

You can imagine each blue circle/face as 1 out of 100 typical students. A student with percentile rank 50 is about in the middle as they have 50 students scoring lower than them. A student performing at the 3rd percentile will have 3 students scoring lower than them.

Most of the students will achieve test scores near the middle of the distribution, with relatively fewer getting the lowest or highest test scores. A helpful analogy is to think of a cross country race or a marathon. If you watch a large race from the finish line, the fastest runner often comes in well ahead of the second fastest. Then, the next few fastest runners will cross the finish line individually, typically with noticeable gaps in between them. Following this, the main body of runners will show up, with many average runners crossing the finish line in quick succession. At the tail end will come the slower runners, once again separated typically by larger and larger gaps between them, with the final runner or two lagging significantly behind the next slowest runner. If your coach challenged you to pass 10 runners and move up your percentile rank by 10 at the next race, where would you rather be? In the back of the pack, the middle, or right near the front?

A runner at the very back of the pack or near the front would need to cut their time significantly more than a runner in the middle in order to pass an equal number of runners. This same logic applies to standardized test results. A student near the very bottom or the very top of the achievement distribution needs to improve their score by a lot more than a student in the middle to result in an equal increase in percentile rank. Herein lies the central reason why any sort of arithmetic done on percentile ranks is invalid: the unit of 1 percentile rank doesn’t exist and isn’t a unit at all. It’s small in some places, and much larger in others. It’s not like all the other numbers in your life where a dollar is a dollar or a degree of Fahrenheit is a degree of Fahrenheit.

What is a principal, education data practitioner or teacher to do when they want to analyze a dataset that only includes percentile ranks? The answer is deceptively simple, and revolves around a statistical conversion developed within the education data world called a Normal Curve Equivalent, or NCE.

It turns out that NCEs are similar to percentile ranks and they are anchored to each other in three places such that 1 NCE = 1 Percentile Rank (PR), 50 NCE = 50 PR, and 99 NCE = 99 PR. Everywhere in between, they have been rescaled or standardized to be what mathematicians calls “equal interval.” This means that growth of 10 NCE is about the same difficulty at all places on the scale. A student growing from the 1st NCE to the 11st NCE is about as noteworthy as from the 50th to the 60th. This is not true for percentiles.

Converting percentile ranks to NCEs is useful in a variety of ways. NCEs allow someone wanting to subtract (find the difference) or average growth scores to do so without making a big analytical mistake. They also allow for an analyst to compare growth based on how challenging it likely was to achieve that growth. To return to the previous table of teachers’ growth scores, if we convert the percentile scores to NCEs, we get the following results.

With NCEs, this table now more accurately shows the relative above average effectiveness of each teacher. Remember that the percentile rank or NCE staying the same for a class of students is what we expect, so a teacher of average effectiveness would be expected to have a zero in the growth column. Each of these teachers appears to be growing students faster than typical, but Teacher A’s growth is the highest. We can now also see that teacher C and teacher D are roughly equally effective at driving student growth. Had we relied solely on percentiles, teacher D would likely have been frustrated and potentially even been set up for mentoring from someone with lower growth data. Additionally, if teachers were rewarded with merit-based pay for certain levels of growth, the NCE growth numbers are a fairer way to assess teachers’ effectiveness across different starting student ability levels.

There is one important caveat to using NCEs. The formula for conversion from percentile ranks to NCEs assumes normally distributed underlying data. The main time to be wary of this assumption is any time you believe a nontrivial portion of students in the tested population (across the state or the nation) are maxing out the test (scoring close to the max score) or scoring close to the minimum score of what you would get guessing on every question. In general, if it’s a valid and age-appropriate test with a large and diverse population to norm on, the percentile to NCE conversion will be approximately correct and certainly better than attempting to do differences or averages on percentile ranks.

United InfoLytics Loves Education Data

While we serve clients in a variety of industries, we are passionate about helping schools and districts understand their data and make valid conclusions from the analysis on that data. The right understanding helps lead to the right decisions, and ultimately this is good for the health of schools and the students they serve. Whether you need help with one-off data analysis or work to integrate your data systems while making analytics easier, we are happy to talk about your goals and where we can fit into your success.

The post Don’t Take Averages on Percentile Ranks—Use NCEs Instead! appeared first on United InfoLytics.

by Nate Mulder

Just before Christmas 2023, the Tennessee Department of Education gave the state a gift in the form of a new, simple way for parents to view the effectiveness of the schools around them. They assigned each public and charter school in the state a letter grade, A to F, and published these grades on their website. As a former teacher and current data specialist, I was immediately interested in how these letter grades were calculated, and what effect they might have on students, parents, and schools statewide.  What I found was a useful dataset, but only if properly understood. I was thankful for the state’s effort to provide parents up-to-date information on school success, but I was equally struck by the opportunity for improvement in the user interface that could enable the public to more easily understand and use the information. When a parent searches for a school on the TN letter Grade Dashboard, they are greeted with the following screen

When I first opened this page, I was overwhelmed by the amount of tables, boxes, and text—and I work in educational data full time! I asked my wife, a former teacher without a data background, to spend 10 minutes on the site and talk aloud about what she thought. Below are some of her quotes from her time exploring:

     “Wow – okay this is overwhelming.”

     “I’m not sure what I’m supposed to be looking at.”

     “What is up with the score ranges?  Why are they different sizes for different letter grades?”

     “I see the A, but again, what does it mean?”

It’s clear that efforts were made to highlight the most important data and make it understandable: explanatory text at the top, bolded “Level 5” headers in the table, and a scale in the lower right to explain how raw scores become grades. However, since the key objective behind these letter grades is to “provide parents with the information they need to be more informed partners in their child’s learning”, it would be useful to take stock of what the current dashboard is useful for and to imagine ways it could be improved.

What it gets right

There is a lot to like about the School Letter Grades dashboard!  The things it gets right include:

• • Measuring both Achievement and Growth.  Both achievement and growth are useful indicators of school success, and measuring them together is more useful than either in isolation.  In 2022 the Department of Education was planning to release letter grades that weighed growth much more heavily and achievement less heavily, but they realized this resulted in schools with 5% and 80% success rates on statewide achievement tests both being able to receive A’s which is not exactly what the public expects when they think of a report card. Conversely, a system which only measured achievement, rather than growth, would incentivize teaching to students on the cusp of the cut scores to be marked at grade level, but do nothing to prioritize the students far below grade level or well above it (See a breakdown of the effects of growth based measurements in Tennessee here).

     

  • Recent, Relevant Information. While Tennessee already has a state report card that provides in-depth information to parents and stakeholders about schools, that report card’s most recent data is from the 2021-2022 School Year and will not be updated for another few weeks or months.  Application season for the Fall 2024 semester begins soon, and having information that reflects school performance from the prior year is crucial for parents seeking to make wise decisions about which schools will be best for their children.

     

  • Response to Public Input: The Department of Education held twelve public forums and over three hundred public comments made and recorded that contributed to the creation of the letter grades dashboard.  The state ought to continue this level of collaboration and response to public feedback.

How it could be improved

• • Tell a Clear Story: The letter grade is meant to give a strong, simple picture of a school’s performance, but the current dashboard includes overwhelming information without a clear flow or priorities.  Using a couple of visualizations and re-ordering information can help guide viewers through a short story of how the school earned its letter grade.

     

  • Start Simple, Reward Curiosity: A visitor to the report card dashboard site should immediately grasp a “one sentence summary” of how the school performed.  Then, they should have the options to explore further in order to understand the specific data if they so choose.
    
    
  • Guide Parents Towards Next Steps:  The letter grade system was designed to help parents choose the best schools for their children, and to drive them to invest in their current schools more effectively.  Including links including a map of nearby schools for comparison, along with other resources the DOE has already created would help parents to access all of the information and tools the state has to offer.

• What could a reimagined TN Letter Grades Dashboard look like with a focus on user experience?  When a parent, teacher, principal, or any member of the public visits the letter grades dashboard, they should immediately know what information to look at, and the data presented should be arranged to tell a story. Here is my prototype for a more user-friendly dashboard.

This dashboard prototype prioritizes telling the story of the four main indicators that contribute to the overall letter grade.  The visualizations of the donut charts prioritize the 1-5 scores earned in Achievement, Growth, College and Career Readiness, and Growth for Highest Need Students.  A visitor’s eye should track first from left to right to see general areas of strength and weakness for that school, culminating in the large letter grade.  Then their eyes should be drawn left to right again across the center of the screen, to digest the most important data point in each category, and the specific score the school earned, along with how close they were to earning the next highest grade up.  Third, the visitor’s eyes should look at the color-coded buttons they can use to explore the four indicators in greater detail, the optional “Tell me more” prompt that will include descriptions of both how each indicator is calculated and why they received different weights in the summative tally, and finally, the visitor has access to links to supporting information, all of which the DOE has already created, that they can use to better understand the school they are investigating as well as compare it to nearby schools they may want to look into for their child.

In sharing these ideas, I don’t seek to discount the useful data that Tennessee is now providing but rather hope to use this as a case study on dashboard design and imagine what an improved dashboard could look like in the future. We are going to be publishing a series of blogs and vlogs looking at Tennessee education data, and we hope you’ll follow along. We love any new data that the state starts releasing, and we are hopeful for what increased transparency surrounding schools does to help parents and students find successful school homes for each child in the state.

The post Grading the TN Letter Grades Dashboard appeared first on United InfoLytics.

To think about the nature of these questions, let’s start with a simple thinking question: I am a teacher, and I have a class of 4 students. Based on their interim assessment data, they are predicted to have scale scores of 20, 28, 40, and 52. The proficiency cut score on this particular test is 38. Which student(s) are most likely to have an impact on my school’s accountability and performance measurement (proficiency or “success rate”), and which student(s) are most likely to impact my own TVAAS (student growth) numbers? If you think about this carefully, you’ll understand why some teachers in some schools are being told to prioritize teaching to and tutoring a subset of students (those close to passing) even though this is unethical.

Below is a video that explains how TVAAS works, why it’s necessary to have a statewide growth metric, and why TVAAS is quite often mis-interpreted. We will finish with a bit of public policy puzzling: How do the unintended consequences of poorly designed and poorly aligned education accountability create perverse incentives for leaders to prioritize some students and deprioritize others?

Answer to the questions at the beginning: the person with a predicted scale score of 40 is most likely to affect the proficiency rate because they are predicted to be “close” to the cut score of 38. On a good day they’ll definitely be proficient, on a bad day they’ll fall below the cut score and not be proficient. But for those predicted to do very well or very poorly, whether they learn a lot or a little in the month before the test, they are almost certainly going to be what they are predicted to be: proficient or not proficient. This is the root of the practice of making “bubble” lists where teachers are told to focus their energies on all the students who are predicted to be within ___ points of the cut score or who are predicted to have a 20-80% chance of passing.

Teaching to students on the bubble list implicitly deprioritizes those with a low chance of passing and those who are certain to pass, and it turns out this strategy works for pumping up a school’s % of students reaching proficiency / reaching the cut score between “Approaching” mastery and “On Track.” And in my opinion, it’s highly unethical. Cut scores are a public policy mistake that was codified nationwide with No Child Left Behind (NCLB). Even though there have been NCLB reforms, none of them have significantly shifted accountability away from cut scores and toward growing all the students in the building as much as possible.

With regards to TVAAS, it turns out that all students have an equal chance to impact the teacher’s TVAAS. Any growth metric or goal necessarily prioritizes the total growth of their whole classroom, which is great. The problem with TVAAS isn’t the fact that it’s a growth measure; the problem instead is that the 1-5 “TVAAS Levels” are widely misinterpreted. TVAAS is still a very good thing because growth-based accountability is the right kind of accountability rather than cut score accountability.  If teachers, building and district leaders, and legislators properly understand the benefits of growth measures like TVAAS which incentivise teaching and nurturing all students, they can start to put greater emphasis on these measures and reward teachers and schools for supporting all of their students, instead of just those predicted to be a few points above or below the proficiency line.

The post The Power of TVAAS, the Challenge of Interpreting TVAAS appeared first on United InfoLytics.

“Are we on track to attain our goals?” This is something that most of us will ask ourselves at times, and it also something we need to be thinking about at work.

This blog is about a type of goal which is cumulative and has a clear deadline. You either surpass your quantitative goal or come short of it when the deadline comes. Sometimes the deadline is the end of an arbitrary accounting period (end of a month or end of a quarter), but other times it’s related to actual events on the calendar which cannot be moved. If your goal is for 1,000 people to register for a conference, your deadline really is fixed as nobody is going to register after the conference is done. Similarly, academic years have a fixed start date, and enrollment goals therefore have a fixed deadline.

You probably already know that a dashboard could help you track your progress. The difficulty is that many dashboards designed for this sort of goal don’t truly help answer the question of whether we are on track to attain the goal. They give a bunch of numbers, which feels like a good thing, but they don’t provide a projection for what we will ultimately attain by the deadline nor do they provide sufficient context to tell you if the current numbers are predictive of success.

For example, if a conference has a registration goal of 1000, here’s the typical basic dashboard. Take some time to consider if this is enough information to determine if we are on track for the enrollment goal:

At first glance, this looks like a good dashboard, and it’s certainly better than no dashboard at all. Some smaller teams are just going to work hard, cross their fingers, and hope for the best without a system for tracking progress, so if you’ve got a dashboard in your organization that looks like this, you’ve already made an important step. What you’re missing in the above dashboard is any sense of trajectory and the current rate of registrations, so now consider this upgraded dashboard:

This is better because we can see the rate of registrations over time. If I was presented with this dashboard, I would say that it generally looks like we are on track. Admittedly the last week or so of registrations has been a bit slower, but we are over halfway to the goal at the halfway point of the registration window. If this is the first convening of this particular conference, you can accurately say, “I’m optimistic the pace of registrations will stay the same and we’ll surpass the goal.” If, however, you have convened this particular conference before, you have valuable historical data that you’re leaving out of your dashboard and you should definitely add that in:

With the addition of three new colored lines, the story changes. Yes, we’re over halfway to the goal at the midpoint of the registration window, but it looks like prior years have consistently shown that the first half of the registration window is the busier part of the window. For whatever reason, we always have a very good first week and a steady first month, but then things tend to significantly slow down until those last-minute registrations come in right before the deadline. The best interpretation of this upgraded dashboard is that we are on track for our lowest registrations of the last 4 years.

While this is initially discouraging, at least we know we are behind before it’s too late to act. We still have time to increase marketing efforts or turn on certain features in software like tracking partial/abandoned registrations for careful follow up. We can even prepare an email blitz the day after the formal registration deadline telling people that we’re taking walk-up registrations at no additional cost this year.

There’s one more thing that a truly effective dashboard might offer: a mathematical projection of where we are likely to end up. While some people can look at the above dashboard and quickly estimate that we are on track for 800-850 registrations, many people cannot easily eyeball this sort of graph and extrapolate how it’s likely to play out. I’ve found over the years that not everyone has the same graph literacy, and it really is true that some people glaze over whenever they see graphs. For this sort of person, you absolutely owe them a clear projection so that when they look at the dashboard, they get some professional assistance in drawing conclusions. Here’s the final dashboard that ensures that all viewers can interpret and draw conclusions from the dashboard:

There are going to be some judgment calls in making the formula that underlies the projection, and there are some complexities that one can add into the formula to make it better, but all of them will look (at minimum) at where we are today vs. where we were on the same day in prior registration windows. If you have an annual conference or annual enrollment cycle and your windows have the same dates each year, it’ll just be where you were at the same time each of the prior years. Regardless of which formulas you use, a clear projection that anticipates underperformance well in advance of a deadline can make people wake up and say, “let’s do something about this while we still have time.” The key as an analyst is to be able to explain how you did the projection mathematically and to show visually that the projection is believable just by walking out the orange line above to the end of the graph. Then you can tell people that every time the dashboard refreshes, they’ll be able to tell if the situation is improving by how that projection moves over time. If tomorrow we click refresh and it shows 840, and next week it shows 870, then we are making some progress toward closing the gap with the goal.

We hope this helps you and your team as you think about measuring progress toward an incremental goal that has a fixed deadline. The dashboard examples shown here are all done in Looker Studio which is one of our recommended best free solutions for dashboarding and is especially great for anyone who uses Google Workspace already. We even will recommend it to someone with Office 365 or a Microsoft-Centric data system under certain circumstances where they only need a specific project dashboard instead of a comprehensive organizational dashboard. To get a feel for interactive Looker Studio features, click around below, especially on the graphs on the right side:

Go and dream of great dashboards! Help your team make the right decisions, draw the right conclusions, and hit your goals. If you want our help jumpstarting your next project, empowering your internal data team, taking an existing project to the next level, or planning a comprehensive data plan or dashboarding project, we would love to talk!

We are planning to cover several other types of goals in upcoming blogs. How you approach determining if you’re on track for attaining a goal depends on the nature of your goal. Some other common types of goals include:

• Performance goals: ongoing goals that can be measured on various different timescales. Examples include attaining or surpassing a certain quality standard, or attaining and maintaining a specified customer satisfaction metric, ensuring a process continues to run efficiently.
• Project completion deadline goals: get something done before the deadline. Success is finishing early or on time, missing the goal is missing the deadline. These are fundamentally different from the deadline goals covered above because you aren’t allowed to leave the project unfinished but you are allowed to miss the deadline as long as you’re willing to deal with the consequences of missing the deadline.

Be on the lookout as we cover these other goals and how to best track them on a dashboard.

Header Artwork Credit: The Amazing Nate Mulder

The post Tracking Progress Towards Cumulative Goals appeared first on United InfoLytics.

Schools had a year to prepare their systems and budget expectations for the new TISA world.. The official TISA guide document the state provides is 54 pages long, but the most important shift is the move from what they call a resource-based funding model to a student-based funding model. Where the BEP funding formula was based on complex formulas drawing on assumptions about resources needed for staffing, textbooks, technology, and many other education costs, relative to their location and other unique needs, the TISA system allocates a base dollar amount per student in the state, with a short list of adjustments.

We will quickly outline TISA, the implications for schools, the importance of using multiple matching methods for databases, and finally our advanced AI- and ML-backed fuzzy matching algorithm engineered to go further than off-the-shelf fuzzy matching. The key takeaway here is that fuzzy matching helps when datasets might have some typos or missing data and in this particular case, fuzzy matching can have a huge return on investment in terms of ensuring schools claim all the funding that students are eligible for. Our advanced fuzzy match can almost always more matches than generic fuzzy matching tools.

The TISA Template

The amount of funding a school receives under TISA starts at a base of $7,075 per student. That base number is then adjusted based on a few important criteria:

+5% per student for very small districts
+5% for rural districts with sparse population
+5% for schools in neighborhoods of concentrated poverty
+15-150% based on special learning needs like Dyslexia, English Language Learners, and other special
education classifications
+25% for each student categorized as economically disadvantaged by certain criteria

The TISA system is intended to be simpler and more transparent for schools than the previous BEP program, but there is a high-stakes catch to the new funding model: school funding depends on data quality and accurately capturing all these special student characteristics. In the past incomplete capture of the above characteristics affected the quality of statistical reports and education research, but now it also negatively affects which funding adjustments students qualify for. If a school or district fails to properly capture these student characteristics, this money which is intended to help support students is effectively forfeit.

The “weights” segment above is reliant on schools ability to understand the
new law and claim their funding

While several of these classifications are pretty easy to handle by marking all students in a “sparse” district as sparse in the student information system, those which vary student to student require more work. One of the ways that students are classified as economically disadvantaged under the funding formula is if they are “directly certified” as receiving public benefits like SNAP (food assistance) or TANF (cash assistance). Currently the state provides districts with a list of directly certified students for each county and relies on schools and districts to match this list to their student roster marking the matched students as being directly certified in the student information system. Doing this well is not as easy as it might seem.

Typically this task falls to someone in the district or charter school data team—someone who can use excel and run a VLOOKUP to attempt to match students on a key identifier like social security number. If every student’s SSN was known and accurate in both the state DHS/SNAP/TANF database and the school student information system, this would result in 100% of matches found in an hour of work. Unfortunately like all databases there will be inaccuracies and missing data in one or both databases resulting in missed matches. A secondary pass through the databases is warranted and typically this will be done based on matching for exact matches on First Name, Last Name and Date of Birth. In the past the decision on how many different passes and different creative matching attempts to make was left to the data team, but today school and district CFOs, superintendents and leaders need to be involved and helping to ensure that the very best efforts are made—possibly 5 or more different passes with different matching criteria and algorithms.
Each additional student matched results in $1,769 additional funding for the school. It is likely that most schools and districts will under-invest in matching efforts by calling it “good enough” a bit too soon. If a more capable matching algorithm could turn up even a single additional match, the return on investment for this additional effort makes it an easy win to spend the time and effort ensuring as close to 100% of possible matches are found. The reality is that most schools and districts will be turning up 10 or 100 additional matches and making this additional effort pay off tens or hundreds of times over.

The direct certification database includes first and last name, date of birth, address, and Social Security Number.  By way of illustration, for a school with about half of students meeting the definition of economic disadvantage, running VLOOKUPs on a variety of different match criteria would typically return the results like these:

A few students may be found through searching for siblings at other schools who appear on the list

Based on our experience, schools that run one or two of the suggested lookups are likely to catch only 80-85% of the total number of students who are eligible for additional funding. The key column in the above graph is the last one showing that no one match criteria is as good as all of them put together—finding all students that match any of the match criteria.

For a school or district with 1000 students in a moderately disadvantaged area, just doing the extra effort of running 4 or more matching rules instead of 1 matching rule can easily turn into finding 70 additional students who qualify for this additional funding—or $120,000.

The Final Step to 100%: The Power of United InfoLytics’ Custom Matching Solution

By examining multiple matching criteria, more matches will be caught despite the typos and missing data in any database. One final tool remains: we can do even better with the power of “fuzzy” matching! You can imagine a fuzzy match as a tool that “squints” at two datasets, and sees if the shape of any row of entries looks similar to another when you blur your eyes a bit. It is tolerant of multiple types of typos and can match two rows even if there is a typo in almost every column. It can rate potential matches by their likelihood and tell you that the following two people are indeed likely the same person:

John Dow, 10/12/1955, SSN: 123-45-6789, Address: 1488 Main Street, Memphis TN 38122
Jon Doe, 10/12/1995, SSN: 123-54-6789, Address: 1488 Maine Street, Memphis TN 39122

The most user accessible off-the-shelf fuzzy matching tool is the one that can be installed for Excel. There is a learning curve to installing it and using it, but it will pay off in terms of additional matches. That having been said, you can do even better than off-the-shelf fuzzy matching tools when you incorporate deep matches with AI and machine learning to calibrate and evaluate even distant possible matches. This is where our custom solution comes in.

United InfoLytics Can Help

We would love to help ensure that you get as close to 100% of the real matches as possible, and our custom AI-backed fuzzy matching tool specifically engineered for DHS to SIS matching is the only tool that consistently outperforms other fuzzy match tools and is purpose-built for the task. We’ve spent over a year iterating on this and we’ve had several breakthroughs over this time that unlock an additional tenth of a percent or half of a percent of additional matches. Now that we believe we’ve engineered a best in class algorithm and software solution for this particular high-stakes funding need that districts have, we are offering it to any district that wants to find what their current matching efforts are missing. We’ve priced it aggressively and fairly: you only pay for what we find and the cost decreases as we find more students. After signing an NDA and a few minutes of your time to share your data with us, we will offer a free scan to tell you approximately how many students we will be able to find for you earn more. Learn more about this service here.

The post TISA Rules and the Power of AI Fuzzy Match appeared first on United InfoLytics.

One note: I use the words algorithm and model a fair bit below. You can think of the algorithm as the procedure used to approach a learning task. The model is the trained state of the algorithm after it works through some training data. A model potentially gets updated in some way each time the algorithm gets new data to learn from. The model is the sum total of the algorithm’s current learning towards the task at hand.

Similar to a human mind without experience, a machine-learning algorithm without training data to learn from is not very useful.

You Be The Machine

To start, I’m going to give you a very quick learning task that is something we might ask a computer to work on. I will have you work on classification of images. Are you ready? You’ve got this! First your training dataset:

Pretend you are the computer. You are given a set of 8 training images each with a label. Look at each of the images as if you’ve never seen these things before; try to learn what a cat is and what a dog is. Note that one good thing about this dataset is that we are teaching the computer that cats and dogs come in a variety of colors/patterns and that these aren’t the defining aspects of the thing. In training a real model, we might give the computer 50 to 50,000 examples of each instead of 4 each, but you get the idea. Now that you have trained yourself on these 8 images, I have a question for you: what are the things in each of the following two images? Before you answer based on your years of lived experience, I want you to attempt to answer as if the only training you have on dogs and cats is the above 8 images.

If you answered “dog” for the first one, great! You have earned your keep as a machine learning algorithm. If you answered “dog” for the second one, I think there’s a chance you cheated and used your life experience instead of just the training data! Look again, and you’ll clearly see that in some ways, the second image looks more like the 4 cats in the training images above. In the training data, all the cats have ears that stand up, they are all sitting upright, and each of the pictures shows their whole body including legs. But each of our training images for the dogs shows just a dog face—no legs, no body.

If you answered “cat” for the second image, I would say that you did your job just fine and there’s nothing wrong with you as a machine learning algorithm. What’s wrong is the fact that our training dataset was very small and lopsided—and thus it biased you, the machine, to think that any furry creature sitting down on its hind legs with pointy ears is a cat. Given that the face in the second image is definitely more doglike than catlike, the model could also be inconclusive saying something like “30% chance of dog, 60% chance of cat, 10% neither.” The facial structure has it thinking it’s somewhat doglike, but the ears and the pose has it thinking more catlike.

Before reading on, I have a question for you: what can you do without writing a line of code to get the algorithm working better to identify cats and dogs in a variety of positions, angles, etc.?

More Data is Good, Diverse Data is Better

You have now learned one of the first pitfalls of machine learning: an algorithm’s ability to label things or predict things is only as good as the training data it is fed. This is honestly true of human learning as well. If you are only exposed to a small slice of the world’s people or the world’s geography, your ability to understand and transfer what you’ve learned to people and places outside of your slice is limited.

Whenever I’m working with a machine learning algorithm, I tend to be exceptionally interested in the nature of the mistakes that it makes. I don’t spend a lot of time celebrating the 95% of things that it handled correctly; I spend my time looking closely at the 5% of things it got wrong, and I think through the main ways I might improve the machine’s performance so that it makes fewer mistakes:

1. Change the algorithm(s) used.
2. Work on ways to code some hints to give the algorithm a leg up on understanding the data—essentially formulas that give it some of the human understanding we have. These hints are called features, and this creative coding work is called “feature engineering.”
3. Work on gathering a larger and more diverse training dataset.

The first two of these are beyond the scope of this article and get a bit technical, but the third one is easy to understand. If we were to give the machine 100 images labeled as cats and 100 images labeled as dogs, it would almost certainly be more skilled at classification than it is with 4 of each. That said, if I still only provide training examples of cats sitting on hind legs and only have dogs’ faces instead of their whole bodies, my algorithm remains limited in its experience and is still limited in its ability to generalize what it has “learned” to a wider variety of situations. It’ll only be truly good at differentiating between sitting cats and the faces of dogs. It would be a more accurate classifier if trained on a reasonably wide set of animal positions, breeds, and backgrounds. Even if I only have time to give it a handful more dog images and cat images, it’ll learn the most and improve the most if those new images added to the training data are diverse. To learn more about AI that struggles in situations outside of its training, see this article on tricks that can beat the best Go-playing AI algorithms.

Admittedly most readers are more likely to be interested in using AI to rank potential customers or classify data in databases rather than classifying images of pets. The principles are the same, however, and building a diverse training set will be important for any business use of machine learning.

Getting philosophical on finding what you seek

In machine learning, it turns out that the most valuable training examples are those that are confusing or are outright surprises to the algorithm when it first sees them. To use human language, most algorithms seek to minimize the “cognitive dissonance” the model “feels” with regards to new confusing cases. The technical term for this is “minimizing the loss” but essentially you can think of it like this: the model changes most when it encounters a training example that is most surprising or most contrary to its current model of the task at hand. It changes significantly in order to ensure it would have a higher chance of success next time if presented with a similar image and asked to classify it.

If you train the algorithm on an additional 100 cat pictures and for all 100 it would have already known they were cats, it might improve just a bit in its abilities but not remarkably. If instead you give it just 20 additional cat pictures and 5 of them were surprises where the algorithm would have initially said, “not a cat,” this additional training data or experience is very impactful and will improve the accuracy of the model significantly—or at least it’ll decrease the model’s overconfidence.

The same is true in life. Your understanding of the world is most improved by new information that you didn’t expect, and it only improves slowly when given additional information that already squares with your understanding of the world. This is why scientists often say that they long for surprises in their experimental data. Data that goes exactly opposite your working model forces you to think and learn a lot more than data that confirms what you already think.

Binary Classification—and when “no” leads to “yes”

Whether it comes to finding the right person to hire, the right next job, the winning needles in a haystack of data, it turns out that closed doors, failures, and mistakes are exceptionally valuable. It turns out that the “nos” in life are the key to the next “yes.” It’s not just a cliche: closed doors are key to finding the open ones and failure truly is only failure if you fail to learn from it.

I was recently working on a machine learning model that does binary classification, which is a fancy way of saying that it looks at a spreadsheet and for each row it tries to figure out if it’s a “yes” or a “no” based on what it has learned from other rows that are known to be “yeses” or “nos”. The model was working better than I had initially hoped, and it was identifying needles in a very large haystack of data that experts were missing with traditional non-ML search and labeling algorithms. I was thrilled that it was working so well, and I then decided to see how it would do labeling on a slightly wider set of real-world data. Since I was also working on creating a training dataset from scratch, I would manually review its predictions on new examples; upon manual review, I would add it to the training dataset with the confirmed label.

Right after I started throwing a wider set of data at it, it was performing absolutely horribly and was mislabeling things constantly. I had asked it to extrapolate only slightly outside of the training dataset, so I thought it would be no big deal, but it was. I manually reviewed perhaps 40 examples in a row that it thought were in the “yes” category but all of them were actually “no.” I started looking at my code to see it I had somehow broken it—after all, it had been working quite well the day before.

It turns out that nothing was broken. I stuck with it and kept saying, “actually that’s a no” to heaps of things that it was labeling as a “yes.” The model started to learn. The next day it was back to finding needles in a haystack with great success. It was only then that I realized that the key to getting back to winning was a whole bunch of mistakes that I fed back into the system so it would learn from them.

Getting started

Whether you are trying to see if machine learning can transform your work, or just reading for the purpose of growing your data skills, be encouraged that the each “no” in life and in machine learning will help you clarify where to be looking for the next “yes.” If you want some consulting as you get started with machine learning, you want a complete outsourced solution for it, or even if you just want to ask questions to see if there is potential return on investment from machine learning in your line of work, then answer is certainly “yes”—I’d love to talk. Set up a time today.

Admittedly, image classification or distinguishing between cat pictures and dog pictures is probably not a critical goal for your work. There are so many other things that machine learning can do well, and in many business applications the examples being learned and later predicted are actually rows from a spreadsheet or a database. See this earlier post for a wider explanation of the types of tasks ML can do well.

The post Machine Learning & Finding What You Seek appeared first on United InfoLytics.

Today, the technology behind these algorithms is accessible to even smaller businesses and organizations, and it gives a real advantage to those who leverage it. In this post, you’ll get an accessible overview of how machine learning works. I also hope you’ll have the beginnings of some ideas for transformation of your work using these tools. I’ll intentionally avoid jargon and replace some technical terms with everyday language—you can learn more about the jargon and field-specific terminology in the other machine learning posts.

Learn How it Works—In Under a Minute

While the math and algorithms are complex, understanding the process is actually pretty easy:

1. A human defines the task for the computer by describing the goal (see below) and possibly also selecting an algorithm. Depending on the tool used, this could be a few minutes of clicking in an AutoML tool or it could be writing code that leverages existing widely-used algorithms.
2. The algorithm is given a training dataset to learn from. This could be a database of pictures alongside text labels of the photos or it could simply be a database or spreadsheet of rows and columns. In the same way that humans learn from taking in the world and from experience, this dataset is the “experience” you want the computer to look at and learn from. After training, the algorithm will have “learned” something from the training data and be ready to apply it to new things. This learning is actually just tweaking a bunch of numbers that often start as zeroes in the algorithm’s initial framework. The numbers are picked so as to best “fit” the “experience” of the training dataset.
3. We apply the algorithm in its post-training “learned state” to some real-world examples that we didn’t let the computer learn from during its initial training. We intentionally held back some real-world examples so the computer couldn’t cheat and simply memorize the answer for each and every example we initially gave it. That’s not intelligence nor is it even artificial intelligence—that’s simply having a good memory. We need to find out at this stage if it has learned something that can be generalized to new situations or if it is actually pretty worthless when it comes to things it hasn’t yet seen.
4. If everything is satisfactory, the trained algorithm can begin to be applied to business or organizational processes—ultimately to working smarter instead of simply working harder.

Essentially, this is not all that different from the way we as humans learn about the world. We are constantly processing and reprocessing our cumulative experience to date in order to be ready to understand things we haven’t seen yet and to make predictions. Interestingly, just as humans can have bias, overconfidence and other similar cognitive blind spots, machine learning has analogous traps and there are important best practices for avoiding and working past them. One of the fantastic rewards from understanding machine learning is the insights it can give us into understanding the process of our own human learning, the reasons we sometimes get things very wrong, and the experiences most likely to propel us quickly to a better and more accurate understanding of the world.

A Real-World Example

Suppose you have 5,000 people who have indicated preliminary interest in what you provide: products, services, education, etc. You may have gotten these names from a form on your website or from people stopping by your booth at a conference/event, or they may be people who have purchased, attended events, or volunteered in the past. Suppose you need 200 people to take a specific action in order to hit your goals. The action could be a purchase, enrollment, commitment, etc. One way to achieve this goal is to blast out an email to all 5,000 people once a week and see what happens. If that only gets you to 75 people, how do you close the gap between 75 and your goal of 200? The natural thing might be to pick up the phone and start calling to build personal connection, explain what you provide, and explain the advantages you offer. Based on past trends, you know that 1 in 20 people you talk to will go to the next step and take the action you are hoping for. You’re going to need to call 2,500 people because if you get 1 in 20, you’ll get 125 from that 2,500, and you’ll close your gap from 75 to the goal of 200. (For more on this kind of math, see the blog post on funnels.) It will take you eighteen 8-hour work days to call 2,500 people on the list and you’ll be done. Have fun! Have fun? Let’s be honest: it’s going to take more than eighteen days because you’re not going to enjoy all that cold calling, and you’re going to want to do it in 2-hour chunks rather than 8-hour chunks.

One day two of calling, you realize that perhaps some of the names on the list are somewhat more likely to be the people you’re looking for. You run an analysis on past data and you find that people who gave their name at an event and already had a brief conversation in person are 30% more likely to take the next step you’re hoping for compared to people who just filled out a web form. This is great news—after crunching some numbers you estimate that if you call these event attendees first, you’re only going to have to call 2,000 names on the list instead of 2,500 in order to hit your goal of 125 more people.

Enter machine learning: let’s take your earlier insight about some people being more likely to proceed compared to others and put it on steroids. It turns out that the predictors of someone being more interested, more likely to take the desired action are multifactorial, which is another way of saying that a number of different things are predictors. In a sales database your predictors might include having made a prior purchase, having spent more than 20 minutes on the website, having asked a question via website chat, proximity to your physical location (ZIP code), and a host of other things. In a prospective student database, it might include number of prior email clicks, how they found out about your school, whether they have responded to prior text messages, whether they have an in progress application, etc.

With machine learning, you create a training dataset and give it to the algorithm. First you filter your database for training data: people who have been in the system long enough that you know whether they were a “yes” or a “no”. Someone who has purchased or enrolled in the past is a “yes” and someone who indicated interest but has become stale (no longer reading emails, no recent website visits) is a “no” because we don’t think they are ever going to become a “yes”. Someone who only indicated interest a month ago isn’t part of the training dataset at all because you don’t know if they are going to ultimately be a “yes” or a “no”. Then you figure out what data you have on these people, and you give the algorithm all the fields you believe might be associated with or predictive of someone taking the next step you are hoping for. These possible predictor columns are called features, and there’s one final column which is the “yes” or “no” column that is ultimately what you want the computer to learn to predict. If you want to think statistics, the feature columns are sort of like the independent variables and the yes/no column is the dependent variable and we want the computer to learn to predict the likelihood of a “yes” from all the other information we have on a person. ML has a number of advantages over classical statistics and logistic regression, especially if the goal is all that matters and you aren’t trying to write up a research paper on the topic.

Note that in situations involving employment or enrollment, you absolutely want to leave out of the algorithm anything that would introduce bias. Leave out columns like date of birth, age, gender, race, ethnicity, and income if you have that data. Leave out anything that might be reasonably associated with these because your algorithm may will bring unacceptable bias into its predictions this way. In sales settings, this is less of a big deal because how you prioritize marketing and outreach is up to you. In settings where you are offering an opportunity like employment or education, you really need to make sure your algorithm doesn’t end up disadvantaging people based on demographics.

Real-World Outcomes

I’ve worked on problems very similar to the example above multiple times, and each time machine learning has been able to create and validate a predictor that really helps focus the time-consuming personal outreach on the people most likely to be interested. Here’s an example of the results from one such analysis:

1. The algorithm identifies the top 150 of 5,000 people as being 1-in-4 likelihood to take the desired action. Compare that to the overall 1-in-20 likelihood. Call these 150 people, and you’re likely to have 37 people taking the desired next step in no time.
2. The algorithm identifies the next 900 people on the priority list as being 1-in-9 likelihood to take the desired action—still way better than 1 in 20.
3. The algorithm identifies the remaining ~4,000 people as being 1-in-35 likelihood to take the desired action.

If you’ve heard of the Pareto principal before, this should remind you of the 80/20 rule. What this ultimately means is that in a few hours of machine learning work, you’ve cut your expected workload from 2,500 calls to just 942 calls. It’s your choice: you can spend 146 hours calling 2,500 people or you can spend 2-3 hours building a satisfactory machine learning model and then spend 55 hours calling 942 people.

There’s also some psychological momentum your team will have when most of the people you talk to on the phone are really interested and 1 in 4 take the next step you’re hoping for versus only getting 1 in 20. It makes the call campaign fun and encouraging instead of annoying. And you end up not wasting the time of people who aren’t all that likely to be interested in what you offer (the 1-in-35 people referenced above). Everybody wins. You can also use machine learning to figure out who gets mailed promotional material given that is not free like an email blast is. Use your mailing budget the best possible way and have a greater proportion of your material actually read and considered rather than immediately trashed upon arrival.

One final note: if your ML solution or ML consultant is ability to provide you with explanability data, you’ll also get some indication of which variables matter most in prediction an outcome. While not absolutely essential, the insights from this usually spawn productive strategy discussions.

Getting Started

If working smarter sounds great, the good news is that it’s never been easier to get started. More and more databases and platforms are coming with some basic machine learning tools built in. For one example and a sense of the effort involved, read the blog post on Salesforce’s AutoML product, Einstein Prediction Builder. At least half of widely-used CRM platforms have something like this although each goes by a different name and is of different quality and capabilities. There’s a real chance that you’re already using a platform that has the possibility of basic integrated machine learning pipelines. There’s still a learning curve to it, but built-in tools like this can be a real advantage even if you need some initial consulting to get it set up or to train your team.

Even if the cloud platforms you currently use don’t have anything built in, there are integration tools that can get you running with machine learning regardless of your current technology platforms—even if your current dataset is as basic as a spreadsheet. You can also test the water at relatively low cost with a one-time offline machine learning analysis of your database. Get all your records/contacts “scored” with a one-week turnaround, and then work through the list to see for yourself the ROI potential in your industry before going all the way to a live and integrated ML pipeline. United InfoLytics is ready to talk about consulting solutions that fit your budget and are sure to provide near-term ROI and cost savings.

Further Reading: Types of Machine Learning Tasks

Artificial intelligence and machine learning both aim to approximate the ways that humans approach certain learning tasks. Without going into the difference between AI and ML, you can rest assured that this is not about creating a machine that approximates or exceeds the whole of human intelligence but instead to use learning algorithms to get computers teach themselves to become good at a specific task that a human has defined in advance. It turns out it’s often better and easier to write computer code that teaches a computer to teach itself than it is to write code where the programmer directly teaches the computer all the human knowledge that applies to the task at hand. The most common types of machine learning tasks include:

• CLASSIFICATION: answering the question “which is this?” or “what is this?” You give it a photo, it returns “dog” or “cat” or “banana.” Or you give it a the name, length, actors, producer of a movie and it predicts the category “thriller” or “drama.” Many algorithms go further and return probabilities of the different classification labels such as “99% chance it’s a dog but 0.7% chance a cat and 0.3% something else.”
• BINARY CLASSIFICATION is a special case of classification where there are only two possible responses. It answers something similar to “is this a yes or is this a no?” Every transaction you attempt to make with your credit card goes through a lightning-fast binary classification system that returns, “legitimate” or “fraudulent” in a few milliseconds. More often it’ll return a percent chance of a transaction being fraudulent rather than just a simple “yes” or “no.” In sales or enrollment, you can using use binary classification to predict the chance that each outstanding sales contact or each prospective applicant will ultimately purchase or enroll.
• REGRESSION: Estimating or predicting an unknown number using known information. This could be predicting a student’s end of year test score or estimating the market value of a home using known data about the student or the home. The main difference from classification is the output being a number not a label or probability of a label.
• CLUSTERING: Identifying similar records in a database, grouping things by similarity. I realize this may sound a bit like classification above but the key here is that we aren’t coming up with labels that coherently describe the class of things in human language—we’re just saying that they are similar. Clustering is one approach to the recommendation engines like you’re used to in online shopping or streaming sites. These sites generally don’t need to be able to put a precise label on a grouping of things as just knowing they are similar is enough to recommend them.
• FORECASTING: Look at what has happened in the past and where things are right now and predict at regular intervals what will happen in the future. Think weather forecasting or predicting the movement of the price of a stock.

These are all things that you and I can do with our human intelligence. Why bring a computer into it when we can do it ourselves? The advantages of machine learning are that it can do these tasks very quickly and sometimes even more accurately than the average human can. The speed factor means we can accomplish more with less effort and not spend time doing work that a computer can do for us. The accuracy advantage, which is not guaranteed but is often possible, means that the computer ultimately gets better than most humans at a very specific task. Even if the algorithm turns out to be 5% less accurate than a human, the speed with which it can approach its work makes it all worthwhile, and the algorithm can refer the situations that it is less confident about to a human for manual examination.

The post Machine Learning for Process and Organizational Transformation appeared first on United InfoLytics.

In each of the simplified example funnels above, think of the width of a funnel at a given stage as the count of the number of people/entities/items still in the process at that stage in the funnel. Then between each stage, illustrated in yellow above, there’s some sort of potential narrowing of the funnel—the narrowing being driven by those that didn’t progress. Not every sales lead wants to talk. Not everyone you talk to wants a quote. Not every student you offer admission to ultimately enrolls. A funnel shows the quantities at each stage in the process as well as the extent to which the funnel narrows at each stage in the process. Funnels also help us by defining success in advance (the bottom stage in the funnel) and focusing us on analyzing each other step in the process in terms of how it affects the final desired outcome.

Let’s talk math & strategy briefly

To think about the math of funnels, consider this scenario: you are the admissions director for a school. This school could be a K12 charter, a K12 district competing with local charters, a college, a university, or a training program of any sort. Last year, you had 3,000 contacts in your database who either indicated interest or were recommended by someone. Of these, 900 applied, and you admitted 600 total. This year your goal is to admit 800, or an increase of 1/3 on the prior year. What are your approaches to hitting your goal? Mathematically speaking you have two general ways to approach this:

• Make the top of your funnel bigger: in this case, we’re going to need more than 3,000 contacts in the database to be emailing, texting, and reaching out to—and we are going to hope that the progression rates within the funnel stay the same as in prior years. If you assume everything else stays the same and you assume that the “quality” of your new contacts is similar to last year, you could hope to hit your goal by getting 4,000 contacts in the top of your recruiting funnel. Increase the top of your funnel by 33% (3000 to 4000) and you can reasonably hope that the bottom of your funnel also increases by 33% (600 to 800).
  Note that in this case, the “quality” of a new contact is basically their likelihood of being interested in your school and ultimately enrolling. If you only widen the funnel by accumulating new contacts that are generally less likely to apply and enroll—people who live a long way from your school or otherwise aren’t as likely to be interested—the additional 33% in at the top of the funnel may only turn into a few percent increase at the bottom of the funnel. There are very data-driven ways to measure & predict progression likelihood even at the initial interest stage using machine learning and predictive analytics, so there are ways to know in advance if the 33% you added to the top of the funnel is likely to lead to a full 33% increase at the bottom of the funnel.
• Increase the progression rates: in this case we are talking about increasing the application rate (out of total contacts), the admission rate (out of total applicants), and the enrollment (out of total admitted students). Assuming that you don’t want to loosen the selectivity of your school, you cannot focus on increasing the admission rate—unless you identify a process issue which is leading to high quality applicants being denied. So you should focus on increasing the application rate of the contacts you have and on increasing the enrollment rate of those you admit to the school. By doing these things, it is still quite possible to hit our goal even without any widening of the top of the funnel.

In reality, you’ll likely pick a strategy that involves both widening the top of your funnel and increasing progression rates through the admission funnel. The situation of the admissions director reasonably allows for both: an increase in new contacts through marketing, an increase in the application rate through with better communication with prospects, and an increase in the enrollment rate through intensive encouragement and engagement for those who are admitted.

Admittedly, some situations don’t allow both a widening of the top of the funnel and an increase of progression rates. Sometimes our knowledge of the situation tells us it’s unlikely that anything we do will change the progression rates in the process—then we simply need to make the top of our funnel bigger. In other situations like an intervention funnel, our goal is actually defined as a success rate out of the cases assigned to us, so adding more in the top of the funnel isn’t an option. Widening the top of the funnel wouldn’t help help attain a goal that is a percent of assigned cases rather than a number of successful outcomes. In this case, the only solution is to increase progression rates through our funnel.

A bit more (very simple) math

Imagine a highly simplified two-step sales funnel: 1000 leads that turns into 500 quotes which turn into 250 sales. Stop and think for a minute about where you would focus your efforts if your main goal was to increase sales. If you only had the capacity to focus on either (a) turning more leads into quotes or (b) turning more quotes into sales, which would you do? Try to make an instinctive decision on this before you read on.

Okay now for the math: at which stage are we losing more opportunities for a sale? Should that determine where we focus our efforts? Some people might say that the stage where we are losing more opportunities is in the progression of leads to quotes: we lost 500 opportunities here where the lead didn’t ultimately want a quote, and we only lost 250 opportunities between the quote and sale stage where someone who got a quote did not ultimately decide to purchase.

It turns out, though, that this isn’t the best way to think about it. It’s better to think of funnels and stage progression entirely in terms of percents. We had a 50% progression rate from lead to quote, and a 50% progression rate from quote to sale. In this sense, each stage has an identical progression rate. If we find a way to raise either (but not both) of the two progression rates to 60%, we would have the exact same final outcome. Put another way, 50% of 60% of 1000 is the same as 60% of 50% of 1000. This is because 1000 × 0.6 × 0.5 = 300 just as 1000 × 0.5 × 0.6 = 300. If you just had the realization that this is the commutative property of multiplication, your middle-school math teacher would be so very pleased.

Strategic Funnel Strategy

Ultimately, deciding where to focus your efforts on a funnel like this comes down to a strategic process that looks a bit like this example. Of course, each funnel and each business has to fill in the last two columns for themselves and it’ll be different in each situation or industry.

After looking at this, most people will come to the conclusion that the easiest and cheapest way to get more sales would be to do more to turn more quotes into sales. A slightly more complex but equally good idea would be to use online advertising to get more leads into the top of the funnel and then to depend on Artificial Intelligence & Machine Learning (AI/ML) to prioritize the leads most likely to convert. This allows us to not hire additional salespeople but instead to focus their efforts on the people most likely to make a purchase. I say “slightly more complex” because it does require some work on the AI/ML front, but the cost of using these advanced technologies is coming down and is entirely within reach for small businesses and nonprofits where it wasn’t as accessible 5-10 years ago.

Funnels Are Your Dashboard

Coming back to the initial thoughts on data-driven operation, operations is about processes and funnels can help us understand processes. Maybe you have a dashboard already for your key operations goals. Maybe you don’t. Either way, perhaps the best way to visualize your dashboard is through a funnel. Why not create (or recreate) your operations dashboard with your key funnels as the organizing principle of your dashboard? The goal of any dashboard should be to know how you’re doing well in advance of a goal or deadline. Wouldn’t you like to know if your sales, recruiting, or intervention pipeline was on track for your goals instead of just waiting for the quarter to end or the recruiting season to close and get the final numbers for how things turned out?

If you want help defining your key operations funnels, ensuring you measure & capture the right stages and progression rates, or creating a live dashboard to know how you’re progressing, United InfoLytics is here to help. Technology and analytics really can help you work smarter instead of simply working harder or hiring more people to continue working in your current paradigm. Let’s set up a time to talk soon and discuss the best ways to define & visualize your funnels. Or let’s talk about using artificial intelligence and machine learning to focus your efforts at each stage in the funnel on those most likely to progress, allowing you to increase progression rates and your final outcomes without hiring additional staff!

The post Operations Goals are Data Goals: Using Funnels To Hit Targets appeared first on United InfoLytics.

Many nonprofits don’t create a logic model until writing a specific grant proposal requires them to do so, and they start researching logic models while facing a submission deadline related to securing new or additional funding. This is definitely a good reason to start learning about logic models, but logic models are so powerful and helpful to aligning and explaining a nonprofit’s work that most nonprofits should start making their first logic model before they even apply for nonprofit status. It’s also a great idea for most nonprofits to be updating their logic model as often as their strategy or activities change because it’s better to have one ready to go when it’s time to apply for funding rather than updating it when facing a deadline.

What is a logic model?

A logic model is an aligned plan for impact. In short, it’s what you need to do your work, the work you do, and the meaningful differences that this work should hopefully lead to for society and the people you impact. Typically the logic model is organized horizontally with terminology similar to this:

The column headings may change slightly from one logic model to another, but your logic model needs all of these concepts. Also, you can visualize the inputs as leading to the activities as leading to the outputs and ultimately the outcomes. It’s called a “logic” model because there’s a logical chain from the inputs to the impacts (outcomes). Ultimately, it’s this logical chain that justifies the inputs: People are unlikely to fund your work or volunteer with you unless they believe as much as you do that the resources given and time volunteered should lead to the outcomes you are seeking.

Alignment Matters

One difference between a perfunctory logic model and a truly useful and self-explanatory logic model is alignment. Essentially, alignment is about whether each item in each column is logically related to items in the immediately adjacent columns. This is especially true if you have multiple areas of work or multiple programs.

Suppose a nonprofit has a summer sports camp and also helps support student athletes academically during the school year. This functionally is two programming streams, and this nonprofit should really be tracking the inputs, activities, outputs, and outcomes of each program all the way across their logic model. If you account for both programs in the inputs/activities/outputs, but you really only attempt to measure outcomes for one of the programs, the logic model will lack alignment because one of your two programs is functionally unmeasured and untracked in terms of measuring impact. Many experienced and data-driven funders will notice this and ask about it—and they are asking a great question! If you had been updating your logic model yearly, you would have noticed this gap and developed a plan for measuring impact right at the moment that your programming and activities changed or broadened.

Once you have a truly aligned plan for your work and measuring your impact, it’s always a good idea to make it beautiful. When you lay out your logic model, try to make it clear and visually appealing so that each program or work clearly flows across and it’s self-evident that your logic model is aligned. United InfoLytics can help with this graphic design if desired.

Logic Models Are More Than a Grant Requirement

The first logic model I worked on was done because it was a grant requirement for a nonprofit I was working at, and I saw it as a compliance task. I was wrong. Building a great logic model is worthwhile for every nonprofit as it ensures you have a plan for what you’re doing that is clear and succinctly articulated. It also helps nonprofit leaders ask questions of themselves like, “why are we doing this particular activity?” or “where would that idea fit into the logic model?” Sometimes trying to clearly outline all activities and all intended outcomes leads us to realize that some aspects of our work are not aligned to the current set of measured goals and desired outcomes. At that point you have to decide to broaden your goals to encompass a wider set of desired outcomes or to prune the activity as it’s just not logical to think of it as actually leading to any of your stated desired impacts.

United InfoLytics Can Help

If you want a data-aligned, funder-ready logic model that’s beautiful and easy to read, United InfoLytics can help. One option for this is to schedule a logic model building session with key members of your team to talk out your current activities, desired impacts, and what data (if any) you currently have on your outcomes. Let’s build a great logic model together, and then let’s make sure there is measurement infrastructure in place to measure results! If this sounds helpful, please take a moment to schedule a time to talk about what this can look like and how data-driven logic model consulting can impact your next steps and funding opportunities.

The post How Logic Models Empower Nonprofits to Increase Impact, Align Data Gathering, and Fund Their Work appeared first on United InfoLytics.