Archive for February, 2008

FAST Search and Enterprise Data Management

Left over from my time at Microsoft, I have a few friends down in Seattle I try and keep in touch with. Last year while I was down at the SOA conference, I met up with my friend who works for a newspaper down there. Specifically he works with FAST Search ( to catalogue and provide retrieval of all past articles from the paper. We had a long chat about its capabilities and its speed (a good thing for a product called FAST). After a while we got onto the idea that an organisation with a large amount of data in various formats and storage mechanisms, say a health organisation with Clinical Information Systems, Radiology, scanned paper records and so on, could utilise a platform like FAST to develop a complete view of their business without going through the usual construction of enterprise data models and repositories.

So in healthcare could we use something like FAST to create an electronic health record system? I wonder. Actually I think it would be entirely possible and might work well in an environment where the decision between indexing and central repository is leaning towards an indexing solution. Instead of developing a publish mechanism where data providers are required to submit events and data pointers to a central index repository, and then developing search algorithms to retrieve the data pointers at run time, why not use an enterprise search engine to crawl the data sources, and use the native functionality of a search engine to index, catalogue and retrieve information for the users? Obviously there are some aspects that would have to be worked on, privacy and security would be high on the list,  but the ability of a search platform to extract metadata (search keys) from both structured and non-structured data is almost perfectly suited to this type of environment.

Development of electronic health record (EHR) solutions is complex and time consuming, particularly on the design phase where, justifiably, all stake holders want to ensure the system provides everything they need in a secure way. The large groups of stakeholders and the long term requirements of an EHR can often mean the projects can take years to define, providing no benefit in the short term. Could an enterprise search engine based solution provide an interim solution for those jurisdictions determined to pursue a centralised data model?