Towards Democratized Academic LLM Hosting
Maart 16 - 10:00 - 17:30
Register
Towards Democratized Academic LLM Hosting
A 4TU Joint Workshop on Local Inference Infrastructure (hosted at TU Delft)
Large Language Models (LLMs) have become core research infrastructure for academia, underpinning research, education, and innovation across disciplines. Across Dutch and European academia, multiple initiatives are currently underway to host, operate, and govern LLMs locally, driven by concerns around data sovereignty, sustainability, cost, transparency, and responsible use.
This 4TU.NIRICT-funded workshop brings together currently siloed academic hosting initiatives across Dutch institutions (eg, Delft, TU/e, VU, WUR, SURF) to foster knowledge exchange, technical alignment, and collaboration around LLM inference infrastructure.
The workshop addresses LLM hosting as a socio-technical challenge, spanning infrastructure and deployment models, energy efficiency, governance and operational sustainability, tooling ecosystems, and integration into research and education. It lays the groundwork for a coordinated, academic-first Dutch ecosystem for responsible, open LLM hosting.
Full Programme March 16
10.00 – 10.30 Welcome and Registration
10.30 – 10.35 Opening Remarks
In this talk, we will show some of the challenges and lessons learned while hosting an LLM inference service. BLABLADOR has been running as a single-man-operation for most of its existence, until very recently.
This talk is more of a conversation, where we will explore themes such as:
- Reliability
- Software Ecosystem and evolution
- From small clusters to supercomputers to cloud: quite a messy journey
- Who can use it and handling user abuse
- Advertising the service
- Choosing which models to offer
- Political consequences of model choice
- How to convince management that it’s worth it
- Present and future of the service
- JAIF and JARVIS: An AI factory for Blablador
- Questions from the audience
Dr. Alexandre Strube
- Maintainer of LMod, the Supercomputers’ module system
- Maintainer of FastChat
- Ubuntu Member, Debian Developer
Dr. Alexandre Strube has been employed at the Juelich Supercomputing Centre for approximately 15 years. During this time, he has engaged in a diverse range of activities, encompassing raw performance modeling, supercomputer benchmarking, and teaching, among many others. Notably, he has been a member of the Helmholtz AI consultants team since 2019, where he has contributed to the democratization of artificial intelligence within academic institutions.
Dr. Strube has been responsible for the development, hosting, and maintenance of BLABLADOR, the LLM inference infrastructure for the Helmholtz Foundation and German Academia, for approximately four years.
11.20 – 11.32 Coffee Break
12.35 – 13.35 Lunch
GPT-NL is a national initiative to build a large language model grounded in the Dutch language, culture, and public values from scratch. Beyond delivering a performant model, the GPT-NL project set out to explore what it means in practice to develop a sovereign language model—one that is transparent, inclusive, compliant with Dutch and European law, and embedded in a broader public-interest ecosystem.
This keynote reflects on the practical experiences accumulated throughout the GPT-NL project in building such a model while simultaneously fostering a diverse and sustainable LLM community. Drawing on lessons learned during data acquisition, curation, and training, the talk will discuss legal and regulatory considerations, approaches to fair and lawful data use, and the organizational challenges of aligning academic, public, and commercial stakeholders. We give special attention to infrastructure-related questions, including large-scale training, hosting, customization, and the role of shared HPC and hosting facilities in enabling broader participation.
Beyond the model itself, the keynote will examine how the GPT-NL project has sought for community building across researchers, LLM and AI developers, data providers, legal experts, infrastructure operators, and application developers—both nationally and in a broader European context. The talk will outline envisioned application domains, strategies for responsible dissemination, and open research directions that arise when a language model is treated as a shared societal asset rather than a closed product.
By sharing successes, open challenges, and unresolved questions, this keynote aims to contribute concrete insights to the discussion on democratized academic LLM community and to inform future national and European efforts toward sovereign, community-centered language models.
Dr. Rer Nat. Julio A. de Oliveira Filho researches and develops the next generation of intelligent autonomous systems—spanning robotics, quantum networks, IoT sensor systems, automotive technologies, and modern Defense applications.
As the leading architect of GPT-NL, the first Dutch-English large language model trained fully from scratch, and one of the creators of NetSquid, the world’s most widely used quantum network simulator, Julio thrives at the intersection of advanced AI, complex systems, and cutting-edge engineering.
He is passionate about elevating AI engineering into a mature discipline—where autonomous systems are not only powerful, but reliable, safe, and explainable by design.
14.20 – 14.25 Room Change
15.25 – 15.40 Coffee Break
16.40 – 17.40 Borrel & Networking
Speakers
Dr. Alexandre Strube
- Maintainer of LMod, the Supercomputers’ module system
- Maintainer of FastChat
- Ubuntu Member, Debian Developer
In this talk, we will show some of the challenges and lessons learned while hosting an LLM inference service. BLABLADOR has been running as a single-man-operation for most of its existence, until very recently.
This talk is more of a conversation, where we will explore themes such as:
- Reliability
- Software Ecosystem and evolution
- From small clusters to supercomputers to cloud: quite a messy journey
- Who can use it and handling user abuse
- Advertising the service
- Choosing which models to offer
- Political consequences of model choice
- How to convince management that it’s worth it
- Present and future of the service
- JAIF and JARVIS: An AI factory for Blablador
- Questions from the audience
Dr. Alexandre Strube has been employed at the Juelich Supercomputing Centre for approximately 15 years. During this time, he has engaged in a diverse range of activities, encompassing raw performance modeling, supercomputer benchmarking, and teaching, among many others. Notably, he has been a member of the Helmholtz AI consultants team since 2019, where he has contributed to the democratization of artificial intelligence within academic institutions.
Dr. Strube has been responsible for the development, hosting, and maintenance of BLABLADOR, the LLM inference infrastructure for the Helmholtz Foundation and German Academia, for approximately four years.
Dr. Rer Nat. Julio A. de Oliveira Filho researches and develops the next generation of intelligent autonomous systems—spanning robotics, quantum networks, IoT sensor systems, automotive technologies, and modern Defense applications.
As the leading architect of GPT-NL, the first Dutch-English large language model trained fully from scratch, and one of the creators of NetSquid, the world’s most widely used quantum network simulator, Julio thrives at the intersection of advanced AI, complex systems, and cutting-edge engineering.
He is passionate about elevating AI engineering into a mature discipline—where autonomous systems are not only powerful, but reliable, safe, and explainable by design.
Sovereign Language Models in Practice: The GPT-NL Endeavour in LLM Community Building
GPT-NL is a national initiative to build a large language model grounded in the Dutch language, culture, and public values from scratch. Beyond delivering a performant model, the GPT-NL project set out to explore what it means in practice to develop a sovereign language model—one that is transparent, inclusive, compliant with Dutch and European law, and embedded in a broader public-interest ecosystem.
This keynote reflects on the practical experiences accumulated throughout the GPT-NL project in building such a model while simultaneously fostering a diverse and sustainable LLM community. Drawing on lessons learned during data acquisition, curation, and training, the talk will discuss legal and regulatory considerations, approaches to fair and lawful data use, and the organizational challenges of aligning academic, public, and commercial stakeholders. We give special attention to infrastructure-related questions, including large-scale training, hosting, customization, and the role of shared HPC and hosting facilities in enabling broader participation.
Beyond the model itself, the keynote will examine how the GPT-NL project has sought for community building across researchers, LLM and AI developers, data providers, legal experts, infrastructure operators, and application developers—both nationally and in a broader European context. The talk will outline envisioned application domains, strategies for responsible dissemination, and open research directions that arise when a language model is treated as a shared societal asset rather than a closed product.
By sharing successes, open challenges, and unresolved questions, this keynote aims to contribute concrete insights to the discussion on democratized academic LLM community and to inform future national and European efforts toward sovereign, community-centered language models.
Add to Calendar
Register
Share this event
