This is a bit long but I find publishing on the blog will allow you to cut and paste as required. As always, feel free to do so.
Development Tools Selection Process
Outlined below is a process for evaluating eLearning authoring/development systems. This same process can be used for a variety of development tools, although the detailed criteria will vary depending on the type of development tool.
Stage 1: Determine Needs
Determine requirements that the development tool must meet:
What is the nature of the training/performance solution that the tool would be use to create?
Examples:
- Information transfer
- Tutorial
- Simulation
- Skill building
- Conceptual
- Cognitive
What are some of the characteristics being considered?
- High level of interactivity
- Engaging
- Ease of use
- Easily accessible
- Available on demand
- Accommodate large learning audience
- Ability to modularize to meet diverse learning needs
- Deliverable to geographically dispersed learning audience
- Ease of interface - keyboard and mouse interaction
- Embedded computer-managed instruction
- Flexible time offered
- Ability to connect to multiple database sources
- Ability to deliver on firm standard hardware platform
- Can be packaged for portable use (vs. requiring participants to take on a special computer in the office)
Stage 2: Conduct Industry Survey
Survey the marketplace to identify candidates. Don't be shy!
Stage 3: Collect Systems Information
Gather information on:
- Delivery platform: What is the delivery platform? Is it multi-platform (in the event of sales)? What delivery media are supported (CD-ROM, networks, etc.) Which operating systems does it run under? Is the delivery platform compatible with our existing technology?
- Development platform: What development platform is required? Is there any special equipment needed? What are the requirements vs. what is already installed in-house? Are there separate development and delivery environments and, if so, what does this entail (e.g., after development, does the program have to be ported over to a different delivery system?)
- Development interfaces: What interface is desired/required? Systems (icons/menus) give the developer more friendly access to development functionality, reduces the learning curve and increases productivity, but at the cost of flexibility and advanced capabilities. A language-based system allows for more complex courseware, but requires substantial training time and has a steep learning curve. Hybrid systems offer an iconic or menu interface along with a language that underlies the system. This dual functionality allows new developers to start developing courses fairly quickly as well as allow for increasing instructional complexity.
- LMS coniderations: What record keeping, tracking/reporting features of participants’ progress does the tool have that is compliant with the LMS?
- Cost and licensing: What are the costs? Are there any runtime license fees? Royalty fees for external sales?
- Training and support: What training and support are available and what is the cost (including any travel to the training)? What consulting services/freelance expertise is available?
- Product stability: How long has the vendor been in this industry? How new is the product? How stable is the product (will it/the vendor be around next years? Three years from now?) How widely is it used in the industry? It would be a bad situation to implement something nationwide, then have the vendor go out of business.
Stage 4: Perform Initial Selection of Several (about 5) Systems for Further Analysis
Based on the criteria above, compare development tools and select several systems that best fit the majority of the requirements.
Stage 5: Down Select to 2-3 Systems for Detailed Analysis
Contact vendors directly to arrange demonstrations of their products and receive an evaluation copy of their software. Assess the systems ease of use, features, etc. Have the vendor demonstrate the system so that the system’s full capabilities can be assessed. Based on demonstrations, data collected, and initial experience, rank systems based on training requirements criteria.
Determine two or three final candidate systems. Selecting more than three increases the evaluation time and makes the final decision more difficult.
Stage 6: Perform Detailed Analysis/Benchmarking on each final candidate
Talk to typical users of the program: What do they like? Dislike? What are the limitations? What is the learning curve? Development cycle?- Conduct benchmark testing to determine each system’s capabilities. Develop a “storyboard” for a sample interaction/design of the system. Use the vendor/someone experienced in the tool, to develop this interaction vs. attempting to develop it yourself to eliminate training time and ease of learning (which should be considered separately outside of these tests).
- Evaluate the following criteria in the benchmarking:- Development or revision time (including time set aside for planning)- Number of authoring screens used in development and revision- System response time between actions or screens for each phase (be sure to test executable on the same machine to eliminate computer configuration variables)- Quality of display in delivery
- Evaluate systems using this criteria by developing sample interaction specification documents in the content of the program to be developed. These documents should contain screen displays and directions for response analysis, branching and feedback. Give these documents to each vendor to develop this interaction in a specific period of time (such as two days).
- Monitor their process, measuring each of the criteria during the specified time frame and then judge the quality of the display during delivery.
- Text abilities:
When developing a piece of elearning, the range of facilities available for text is important, since this is likely to be the primary means of communicating with the student. The range of options available for text is likely to give you a good indication of the overall flexibility of the system. - Graphics
Graphics used in programs can either be generated by the application itself, or imported from files. - Sound
Since future applications may utilise sound, the abilities of the authoring environment to control sound is important. With interactive-audio drawn from a CD, the ability of the system to interface with the necessary software required to control the CD player will be fundamental. Sound can also be supplied from the hard drive. - Software friendliness
Where the program is to be developed/updated partially in-house, the ease of use of the software development environment is important. This reduces the need to subcontract the development of elements of the program to others outside of your organisation. The more complex the software is to use, the more technical expertise will be needed to develop and maintain the program. - Design assistance
When developing the applications, the more tools there are to assist the author, the quicker it will be to develop systems and modify them. - Learner control
The software environment must be capable of allowing the development of systems which allow the inclusion of sophisticated learner control. - Question handling
The ease of programming interactions is also a pointer to the overall abilities of systems. The range and variety of these are important, as is the ability to modify, add or remove interactions from within a program. - Branching
Most applications developed require branching. The route taken by the application should be capable of being decided in a number of ways. - Other user resources
Ability and ease in adding other resources to an application, such as context-sensitive help screens, dictionaries or glossaries, a built-in calculator, student note-taking resources, etc. - Printing options
Printing is not only required for reporting the student progress, but also for documentation and retention purposes. - Documentation and tutorials
Consider how well put together the paper-based documentation and other support for the product is. - Hardware and software
The software should be capable of running on a range of PwC hardware.
Stage 7: Review Detailed Criteria & Recommend Authoring System
Detailed Criteria Program
Finally there are all the little detailed, but important things you need to consider:
- Text abilities
Variety of fonts, sizes & colors
Variety of styles, bold, italic, underline, highlighted
Variety of formats, centered, justified, double spaced
Ability to import text from other sources
Automatic re-formatting of text on the screen
Search & replace ability - Graphics development capabilities
(circles, lines, boxes, etc)
Transition effects (wipes, fades, dissolves, etc)
Color graphics abilities/range of colors
Animation abilities
Generated by the application
Ability to import from other programs
Ability to import graphics in several formats - Sound
Use computer-generated sounds (tones, beeps, etc)
Use sound cards
Utilize sound stored on other media (digital files on hard-disk, CD-ROM) - Software friendliness
Ability to structure layout of learning sequence
Ease of modification of the structure of the program
Ability to change screen layouts quickly
Ease of programming
Availability of developers
Ability to link to other applications
Ability to call software written in another language
Interface - Design assistance
Range of pre-written structures available
Ability to reuse code within other programs
Integral flowcharting of the application
System checks for incomplete branches
Ability to track testing of the system to help debugging
Ability to run sections of program, to aid debugging - Learner control
Pacing & sequence control by learner or program
Able to provide gradual learner control
Learner can go forward or backwards within a lesson
Programmable learner control keys - Question handling
Ease of creating student interactions
Ability to modify, add or remove interactions
Variety of question types
Variety of answer matching, key word, exact, spelling errors
Able to use questions with several correct answers
Able to generate specific feedback for each student answer
Random generation of questions or feedback - Branching
Unlimited branching for menu choices
Able to branch based on correct/incorrect responses
Able to branch based on elapsed time
Able to branch based on number of tries - Other user resources
Easy access to built-in, context-sensitive help screens
Easy access to available dictionaries or glossaries
Access to built-in calculator
Access to student note-taking resources - LMS considerations
Student management/CMIManagement easily turned on or off for specific program
Able to record small or large amounts of data for each student
Able to store elapsed time or level of achievement for each student
Course restart options available to LMS
Able to generate reports for each student or each lessonOutput available as an CSV file - Printing options
Able to print: program text or graphics, branching information, program documentation,
all or part of program - Documentation & tutorials
Well-indexed reference manual included
Tutorial included
Telephone support available
External training available - Hardware & software
Memory (RAM) requirements for delivery
Memory requirements for authoring
Required disk space
Processor requirements
Ability to run on organisational supported hardware
Need for other software (such as Windows)
Need for other hardware (graphics card)
Vendor stability/Support available from software distributors & system developers for continued viability/support
I know this is a bit detailed. Maybe this is why it is called back to basics. How much of this do you take for granted?
No comments:
Post a Comment