By Robert Tai Chiang Vun (robert@might.org.my) and Dr Umi Kalsom Yusof
(umi.kalsom@might.org.my)
In the rapidly evolving landscape of artificial intelligence (AI), knowledge retention
remains one of the most challenging yet crucial aspects for organisations that
depend on expertise, research and continuous innovation. One such organisation,
the Malaysian Industry-Government Group for High Technology (MIGHT), has
recently embarked on a promising initiative using Large Language Models (LLMs)
as a means to capture and preserve tacit knowledge.
This article shares MIGHT’s unique approach in building a proof of concept (PoC)
to address a long-standing issue of how to retain the accumulated knowledge
of over 30 years in a systematic and accessible manner, potentially reshaping
knowledge retention in the industry.
Presentation of PoC at Maistorage Technology Sdn Bhd Grand Launch
Understanding the Organisation
MIGHT is a government-affiliated think tank under the purview of the Ministry of Science, Technology and Innovation (MOSTI). Its primary function is to conduct evidence-based policy studies, generate analytical reports, and provide strategic recommendations aimed at advancing Malaysia’s high-technology sectors. Over the years, MIGHT has made significant contributions to several national-level policy frameworks, most notably the Industry4WRD strategy, which outlines Malaysia’s approach to Industry 4.0 transformation. MIGHT’s role in shaping such policies includes conducting foundational research, stakeholder consultations, and synthesising multidisciplinary insights into actionable recommendations.
These contributions rely on the accumulation of both explicit and tacit knowledge, including datasets, research papers, policy documents, as well as the experiential insights and contextual understanding possessed by subject matter experts. However, like many knowledge-intensive institutions, MIGHT faces a critical organisational challenge in retaining and transferring institutional knowledge. As experienced researchers retire or transition out of the organisation, there is a tangible risk of losing valuable intellectual capital. This challenge emphasises the necessity for formal knowledge management frameworks and institutional mechanisms to ensure continuity and sustained policy impact. Addressing this gap is essential for preserving institutional memory and maintaining strategic capacity in national policy development.
The Problem Statement
At MIGHT, valuable knowledge exists in three forms:
Structured data – such as statistics on GDP, trade values and other quantifiable metrics used in reports.
Unstructured data – including textual content from reports, studies and reference materials.
Tacit knowledge – informal, experience-based insights possessed by researchers and domain experts.
Tacit knowledge, unlike structured data managed through systems such as SharePoint, is challenging to capture and transfer. With the increasing turnover of experts, MIGHT risk losing valuable institutional insights. To address this challenge, a PoC was developed using large language models to systematically retain, organise and make expert knowledge accessible. The goal is to explore the potential of AI in capturing the depth of human expertise, including context, experience and nuance, to make it reusable across the organisation. This initiative aims to improve the preservation and sharing of strategic insights in a knowledge-driven environment.
The Genesis of the Proof of Concept
At the beginning of exploring LLMs, the concept was still quite unfamiliar within MIGHT. Although there was a general awareness of AI, the practical use of fine-tuned language models had not yet been tested. Therefore, MIGHT set out to create a proof of concept that has two main goals: (i) develop an AI-powered tool to help researchers conduct policy reviews, and (ii) explore the possibility of acquiring tacit knowledge from researchers during the LLM training process. Through close collaboration with industry partners, MIGHT started developing a finetuned LLM combined with Retrieval-Augmented Generation (RAG) that could answer questions about previous work by learning from existing documents and stakeholder inputs, acting as a digital expert capable of capturing not only facts but also the contextual insights that shape them.
The Approach: A Systematic Framework
MIGHT’s knowledge retention framework in the PoC consisted of four core components:
This iterative cycle of data collection, Q&A generation, fine-tuning, and validation was repeated until the model’s performance reached optimal levels.
Discovering the Hidden Gold: Capturing Tacit Knowledge
One of the most valuable outcomes from the PoC emerged during the development and validation of Q&A pairs. Although this phase was time-consuming, it appeared to be an effective way to capture tacit knowledge. As domain experts reviewed and refined the model’s responses, they contributed not only facts but also their reasoning, judgment and contextual understanding, which often provide undocumented insights. In doing so, the process unintentionally became a structured method for knowledge elicitation. What started as an attempt to train a language model ultimately provided a systematic way to extract and preserve expert insights, making them easier to access and reuse.
System Architecture and Tools
The system was initially developed using Phison’s on-premises infrastructure to ensure security, scalability and complete data sovereignty. Model training and inference run on the Pixelspace AI Server E10 Series, equipped with two NVIDIA RTX 4090 GPUs (24GB each), enabling full fine-tuning of 7B and 14B parameter open-source LLMs. The aiDAPTIV Pro Suite GUI streamlines the fine-tuning process with an intuitive interface designed for policy review tasks. The setup includes an industry-standard retrieval library for relevance scoring, ensuring precise context matching. Side-by-side comparisons of model outputs before and after tuning allow for continuous performance evaluation and refinement.
Early Results and Lessons Learned
Although still in its early stages, the PoC has shown encouraging results. Some of the responses generated by the model scored high on relevance and accuracy, validated by SMEs. But more importantly, it opened a new door to codify and retrieve organisational wisdom systematically. Here are the key lessons learned:
1. Tacit knowledge can be partially captured.
While we can’t expect to record every nuanced thought of a researcher, a structured Q&A validation process can approximate it.
2. Tooling and infrastructure are mature.
Contrary to earlier fears, the process is not rocket science. Many open-source tools and pretrained models are now readily available for experimentation.
3. Validation is critical.
Without expert validation, fine-tuning is just guesswork. The loop between model development and SME input must be tight and ongoing.
4. User adoption is a challenge.
While SharePoint and SQL databases exist, they are often underutilised. A conversational AI feature might encourage higher engagement among staff.
Future Directions
The journey thus far is paving the way for more ambitious goals, including:
• Building a centralised knowledge repository infused with both structured and tacit knowledge.
• Expanding Q&A coverage to other thematic areas within MIGHT.
• Enabling conversational search through chatbot interfaces.
• Integrating external data sources for dynamic insights.
There is also potential to explore multi-lingual models, given that policy work in Malaysia often involves Bahasa Melayu and English.
Final Thoughts
MIGHT’s PoC on AI has demonstrated the possibility of systematically recording, refining, and retrieving tacit knowledge infused by experienced researchers. This is not just a use case at MIGHT, but a blueprint that other government agencies and knowledge-driven institutions can learn from. As Malaysia continues to grow its digital and AI ecosystem, such initiatives are timely, relevant and deeply necessary.
