Linux Saloon 69: Self-Hosting Our Own Robot Overlords

This Open Mic Night acted as a carryover from last week’s Application Appetizer just a bit as we had some extensive conversation about VIM. Colin starts out the night with his experience using openSUSE with KDE Plasma which, of course, brought a huge smile to my face.

If you have an suggestions for topics, be they news, distributions, applications or anything that is Linux, tech or open source related, comment below or send an email to

Thanks so much for your continued support in watching, sharing and subscribing to Linux Saloon.

Discuss here on the Tux Digital Linux Forum:

00:00:00 Introductions
00:01:51 50 Years of Ethernet
00:02:44 Jinda talks about MX Linux
00:11:05 StrawPoll – Double Click vs Single Click
00:37:34 Self-Hosting AI
01:52:07 Next Week – On Location at South East Linux Fest
01:54:17 Last Call
01:57:49 Bloopers

Live demos (select “most likes”):
Hardware requirements
State of AMD compute:
Self-hosted text to art with stable-diffusion
Rent from a service:
Live demo:
Self-hosted image “dragging”
Live demo:
Self-hosted text chat (top of huggingface leaderboard) (32k context size)
Live demo:
Live demo:
Live demo:
Self-hosted chat with documents
Live demo:
Self-hosted speech-to-text
Live demo:
Self-hosted text-to-speech
Recommended channels

Quick history of recent developments
MosaicMl’s MPT-7B
$300,000 estimated into weights (TO DO: provide link to source)
Based off WizardLM
Limitions of LLM context windows
Token comparison (top 3)
OpenAI ChatGPT – 4,096 tokens
OpenAI GPT-4 – 8,000 tokens (and a 32,000 token version that isn’t publicly available)
MosaicMl’s MPT-7B – 65,000 tokens
What you can do with 65,000 tokens that you can’t with 8,000
Samsung employees accidently leaking trade secrets to OpenAI
Hardware & OS requirements for self-hosting
Model quantization for running on GPUs and CPUs
AMD GPUs aren’t built for AI
cuda cores, drivers, software
Open source AI lacks good UIs
Self-hosting Stable Diffusion demo
Ask for help and you’ll get it, TuxDigital forum or Linux Saloon Telegram
Currently GPU cores best perform the necessary tasks for these locally run AI programs, but as we see in almost every smartphone now, hardware dedicated to AI processing (ML/NE Cores) is a long-term necessity. Any thoughts on how Linux might address hardware ML cores in the future as they come to x86 chips?

Dr. Ian Malcolm: Yeah, but your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.

Other Resources:
Linux Saloon Community on Telegram –
Discord Server –

Robot graphic taken from PETSCII Robots game

Leave a Comment

Start the discussion at

Hosted by: Nathan Wolf

About Linux Saloon

This Week in Linux (TWIL) is the Linux News podcast that will keep you up to date with what’s going on in the Linux world and Michael will give you his take as a 20 year plus Linux user. Join other TWILLERS every Saturday with Your Weekly Source of Linux GNews.

More Episodes

Related Podcasts