ILF Vendor Evaluation Standard
Handing you the tools to evaluate any LMS through learners' eyes
Most learning platforms are built, chosen, and implemented from the wrong side of the screen. The ILF Vendor Evaluation Standard hands you a complete, research-backed method for evaluating any platform so you know if it helps learners or gets in their way.
Most platforms are judged from the wrong side of the screen
Whether you chose your LMS five years ago or you're choosing one next year, the same thing usually happens. People look hard at the screens they'll use themselves: admin, reporting, course building. Almost nobody looks closely at what a learner meets, from the day they enrol to the day they finish.
And that's where the platform does its real work, mostly by default.
It's rarely anyone's fault. Platform decisions go wrong in the same few ways:
-
The wrong screens get judged. The learner's journey gets a quick look, if any.
-
Nobody tests it with real material. A sales demo uses polished sample content. A platform you already run gets judged on habit and complaints. Either way, you never see what it does with your material, from your learner's side.
-
Everyone wants something different, and finds out late. Learning designers, educators, IT and finance each weigh different things, and those differences usually surface after the decision, not before.
What the platform does by default is what your learners end up with.
Your decision: Keep it, fix it, or replace it
Every LMS evaluation ends in one of three decisions. You keep your platform because it's genuinely serving your learners. You fix it, by switching on what you're not using, changing defaults, or taking the gaps to your vendor. Or you replace it, knowing exactly what the next one has to do better.
The ILF Vendor Evaluation Standard gets you to that decision with evidence, not opinion. Your panel tests what the platform actually does, on your own material, at every stage of a learner's journey. Then you weigh it up alongside cost, technology fit and support, using priorities you agreed before anyone looked at a screen.
Your files. Your panel. Your priorities.
How it works: 4 steps, from first question to clear decision
Step 1: Prepare
Bring together your panel of three to five people, with at least one each from learning design, teaching, and technology. Agree what matters, and how much, before the evaluation begins.
Step 2: The platform shows its work
Every platform is tested the same way, whether it's the one you run now or one on your shortlist. Same brief, same files, same panel. Each session opens with the learner's journey, end to end, before anyone looks at an admin screen.
Step 3: Your panel tries it
Your panel goes hands-on in the platform, then confirms in writing what a demonstration can't prove.
Step 4: Decide, with your eyes open
Each evaluator scores alone, then the panel agrees. One workbook brings the learner results together with cost (set-up, running and leaving), technology fit and support, using the weights you agreed at the start.
Three defined levels that everyone reads the same way
'A'
The feature exists
The feature is there, ready to use
'AA'
It's done well
It works the way good learning design needs it to
'AAA'
The system does it for you
Embedded at systems level, good practice is consistently applied regardless of who the user is
If you've worked with standards, you'll recognise the pattern. That top AAA level is the whole idea of the Intentional Learning Framework™ in one line: the system should make good practice the easy path.
And there's no magic number at the end. Results read in plain words ("Meets target", "Below target", "Core feature missing: your call"), so you decide what each gap means for your organisation.
Research-backed, ready to run
The LMS Evaluation Standard is your toolkit featuring seven documents, built from the ILF Specification Papers (500+ pages of research-backed method) and a decade of hands-on research with real adult learners and educators.
For your vendors
-
Demonstration Brief. What each platform must show, with a running order and a technical note so sessions run smoothly.
-
Pre-demonstration Questionnaire. What exists in the platform, in plain questions.
-
Written Confirmation Questions. Sent after the sessions, to confirm what a demonstration can't show.
For your panel only
-
Evaluation Pack. The step-by-step guide and the Field Sheets your panel uses in every session.
-
Scoring Workbook. Each evaluator's scores, panel agreement and plain-language results, one per platform.
-
Decision Workbook. Your criteria and weights, deal-breakers, cost, and a side-by-side comparison that prints on a single page for your CEO or board.
-
Reference Rubric. The full detail behind every line, for briefing the panel and settling disagreements.
-
162 evaluation items
-
6 learner-journey stages, plus Foundations
-
89 vendor questionnaire questions
-
68 written confirmation questions
Built for everyone with a stake in the decision
Leading the platform decision
Make the call fairly, and show that you did. Every platform tested the same way, and everything you need to decide in one place.
Learning designers and educators
Your expertise gets a real seat at the table. Find out whether the platform helps you teach or fights you, and what could change if you knew where to look.
Technology and IT leads
A structured process with clear steps, time estimates and a preparation checklist. Every claim is cross-checked, and your criteria (integrations, security, data hosting, migration) sit openly in the same workbook.
Executives and boards
A defensible decision on a large, long commitment, with a one-page comparison and a documented method that shows exactly why you kept, fixed or replaced.
Buying in the age of AI
AI features tested on what they do, not what they promise. Anything a platform writes for you is checked: does it ask before it drafts, does it invent facts, does it pretend to be a person? Human judgement stays human.
This is already at work, on both sides of the table
This isn't a someday idea.
A technology company has already decided to retire its LMS and is actively building its next 'learning facilitation system' from the ground up on the Intentional Learning Framework™.
A large national education provider has built the ILF into how it evaluates its next platform, and the ILF Evaluation Standard is a key decision tool.
When the people buying platforms and the people building them work from the same standard, learners are the ones who win.
For technology providers
Building or selling a learning platform?
Build to the standard your buyers will be testing you against.
-
Starting fresh? Build from it. The Standard shows your product team what "done well" and "the system does it for you" look like, feature by feature, from the very first sprint.
-
Already in market? Test your own platform before your next pitch. Find the gaps on your terms, and design them out before the demo, not explain them during it.
What the LMS Evaluation Standard doesn't do
The Standard doesn't assess regulatory compliance, accessibility conformance (WCAG), proctoring or privacy law. It doesn't certify a platform, and it gives no single total score. It supports your decision. It doesn't make it for you.
How to get it?
Embedded as part of every full ILF Specification Papers licence
The ILF Vendor Evaluation Standard is licensed as part of the full set of seven ILF Specification Papers, for both Education Provider and Technology Provider licences. The Papers describe what good learning design looks like. The Standard checks whether a platform helps your people do it, or gets in their way.
Licences are priced by organisation type and scale. Each organisation's copy is marked for that organisation.
If your platform is on your mind, whether it's up for renewal, on your shortlist or on your roadmap, let's talk about where you're up to.
FAQ
-
We're not replacing our LMS. Is this for us? Yes. Plenty of providers use it to test the platform they already run: to see what it's doing to their learners' experience, find what they're not using, and take clear, specific gaps to their vendor. It's also a strong way to prepare for a contract renewal.
-
Do we need to use all of it? No. You decide which parts apply to your organisation, and the parts that don't apply simply drop out.
-
Will vendors see our criteria? No. Vendors receive the Demonstration Brief and questionnaires, which ask them to show capabilities. The rubric and scoring stay with your panel.
-
How long does an evaluation take? As a guide, a platform demonstration runs to about six and a half hours (best as two half-days), and hands-on testing takes about 15 hours across several days. Both shrink as parts that don't apply drop out.
-
Can we buy the LMS Evaluation Standard on its own? No. It's built from all seven Papers and draws on every one of them, so it comes with a full ILF Specification Papers licence.
-
Is it only for VET? Definitely not. The language is deliberately sector-neutral, so it works for VET, higher education, workplace training and community organisations alike.
-
Does it certify a platform? No. It supports a decision. A vendor holding a Papers licence is not the same as a platform passing an evaluation.
"Dr Deniese Cox's Intentional Learning Framework" is a registered trade mark in Australia.
"Intentional Learning Framework," "ILF" and "Built with ILF" are used as trade marks of Dr Deniese Cox.