Building An AI Personal Assistant

Getting the Most Out of a Personal AI Assistant

A practical guide for setting up your own AI-powered personal assistant


What Is a Personal AI Assistant?

Think of a personal AI assistant as a knowledgeable friend who never forgets what you’ve told them. Instead of one general-purpose chatbot, a well-built personal assistant is a team of specialized advisors — each one tuned to a different part of your life — backed by a knowledge base of facts, preferences, and context that’s unique to you.

The more the assistant knows about you, the more useful it becomes. A generic chatbot might give you decent restaurant suggestions. Your personal assistant, knowing you live in Old Fourth Ward, that you and your wife prefer walkable spots, that you’ve mentioned a fondness for Thai food, and that you’re retired and flexible on timing — gives you a recommendation with directions, a note about the quieter lunch hours, and a reminder that the place is dog-friendly for the patio.

That’s the difference between a tool and an assistant.


How the System Works: A Plain-English Overview

Agents: Your Team of Specialists

An agent is a specialized advisor with a defined area of expertise. Rather than one AI that tries to know everything, the assistant routes your question to whichever agent is best qualified to answer it.

Here are examples of the kinds of agents a personal assistant can include:

Agent What It Handles
Local Guide Neighborhood restaurants, events, venues, services
Culinary Advisor Recipes, cooking techniques, meal planning
Garden Advisor What to plant, when, pest control, zone-specific advice
Travel Advisor Trip planning, itineraries, destination research
Career/Business Advisor Professional decisions, networking, retirement transitions
Music Advisor Discovery, playlists, concert information
Health & Wellness Advisor Exercise, nutrition, age-appropriate considerations
Financial Advisor Budgeting, planning, professional referrals (not investment advice)
Community Advisor Local organizations, volunteer opportunities, civic engagement
General Advisor Cross-domain questions that touch multiple areas

You don’t need all of these on day one. Start with three or four that match your daily life, and add more as you see the value.

The Knowledge Base: What Makes It Yours

The knowledge base is the collection of facts, preferences, and context documents that the assistant draws from. This is what separates a personal assistant from a generic chatbot.

Knowledge falls into a few categories:

  • Personal profile — Who you are, where you live, your household, your interests
  • Preferences — Cuisines you like, how formal you want responses, neighborhoods you frequent
  • Domain knowledge — Detailed reference material for specific agents (e.g., your garden zone, your dog’s breed and temperament for pet-related questions)
  • Active context — Upcoming trips, current projects, pending decisions
  • Resource guides — Professional contacts, local services, trusted vendors

Skills: What Agents Can Do

A skill is a specific capability attached to an agent. The Culinary Advisor, for example, might have skills for recipe formatting, weekly meal planning, and cooking technique tutorials. Skills define not just what an agent knows, but how it structures its responses for particular tasks.

Routing: How It Knows Who to Ask

When you type a question, the assistant analyzes your words and routes the question to the right agent. Ask about “a good Thai place near the BeltLine” and it goes to your Local Guide. Ask “how do I make pad thai” and it goes to the Culinary Advisor.

This happens through keyword matching, weighted scoring, and conflict resolution. The system recognizes that some words are ambiguous — “restaurant” could mean you want a recommendation (Local Guide) or a recipe inspired by a restaurant dish (Culinary Advisor) — and uses surrounding context to make the right call.

Disambiguation: Handling Ambiguity

Some terms mean different things depending on context. The word “French” could relate to cooking, travel, or language learning. The assistant maintains disambiguation rules that look at the surrounding words in your question to figure out which agent should handle it. If you say “French bistro recipe,” that’s culinary. “French visa requirements” is travel. When it genuinely can’t tell, it routes to a general advisor that can synthesize across domains.

Guardrails: Knowing Its Limits

A responsible assistant knows what it shouldn’t do. Guardrails define boundaries:

  • It won’t give specific medical diagnoses or legal advice — it will point you to professionals
  • It flags when a question needs a human expert (attorney, financial planner, doctor)
  • It respects privacy and doesn’t share information across contexts where it shouldn’t
  • It’s honest about uncertainty rather than making things up

Building Your Knowledge Base: A Checklist

This is the most important part. The better your initial knowledge base, the more useful the assistant is from day one. Set aside an hour and work through these questions. You don’t need to write essays — bullet points and short answers work fine.

Personal Profile

  • Full name and what you prefer to be called
  • Where you live (neighborhood, city, state)
  • Household members (wife’s name, preferences she’s shared with you)
  • Pets (name, breed, age, temperament, any special needs)
  • Your general daily routine and schedule flexibility (retired, part-time, etc.)
  • Any mobility considerations or physical preferences (walking distance, stairs, etc.)

Interests and Activities

  • Hobbies and how active you are in each
  • Community involvement (organizations, boards, volunteer work)
  • Sports or fitness activities
  • Entertainment preferences (music genres, TV, movies, reading)
  • How you prefer to spend a typical weekday vs. weekend

Food and Dining

  • Favorite cuisines and dishes
  • Dietary restrictions or preferences (yours and your wife’s)
  • Favorite local restaurants (and what you love about them)
  • Cooking skill level and what you enjoy making at home
  • Any food-related goals (learning a cuisine, eating healthier, etc.)

Home and Garden

  • Type of home and outdoor space
  • Gardening experience level
  • What you currently grow or want to grow
  • Home maintenance services you use or need

Travel

  • How often you travel and preferred trip style
  • Favorite past destinations (and why)
  • Places on your wish list
  • Travel constraints (budget range, health considerations, pet care logistics)
  • Airline/hotel loyalty programs

Local Knowledge

  • Neighborhoods you frequent besides your own
  • Favorite local businesses and services
  • Trusted professionals (doctor, dentist, mechanic, handyman, etc.)
  • Community events you regularly attend

Communication Preferences

  • How formal or casual should responses be?
  • Do you want brief answers or detailed explanations?
  • Should the assistant proactively suggest things, or only answer when asked?
  • Any topics you do not want the assistant to bring up?

What You Can Expect from Different AI Models

If you use a personal assistant framework, it will likely run on one of the major AI models. Here’s what to expect from the big three, in plain terms:

Claude (Anthropic) tends to give nuanced, well-structured responses. It’s good at following complex instructions and respecting boundaries. It generally avoids making things up and will tell you when it’s unsure. Responses tend to be thorough without being verbose.

ChatGPT (OpenAI) is conversational and approachable. It’s strong at generating creative content and brainstorming. It can sometimes be overly eager to please — giving you a confident-sounding answer even when it should express more uncertainty. It’s very good at explaining concepts in simple terms.

Gemini (Google) excels at integrating with Google’s ecosystem (Maps, Search, Calendar). It’s strong on factual lookups and current information. Responses can sometimes feel more like search results than conversations. It’s a good choice if you’re already deep in the Google ecosystem.

None of these are perfect. All of them occasionally get facts wrong, especially about local businesses, hours, and recent events. A good personal assistant setup mitigates this by injecting your verified knowledge into the conversation, so the AI draws from your facts rather than guessing.


The Honest Truth About What This Can and Can’t Do

It can:

  • Remember everything you tell it and apply it consistently
  • Give personalized recommendations based on your actual preferences
  • Help you plan trips, meals, projects, and activities
  • Draft emails, letters, and other writing in your voice (once it knows your voice)
  • Organize information you’d otherwise keep in your head or on scraps of paper
  • Get better over time as you correct it and add knowledge

It can’t:

  • Replace professional advice (legal, medical, financial)
  • Access the internet in real-time unless specifically configured to do so
  • Make phone calls, book reservations, or take action on your behalf (yet)
  • Read your mind — it only knows what you’ve told it
  • Guarantee 100% accuracy on any factual claim

Getting Started: Three Steps

  1. Fill out the checklist above. Even partial answers make a difference. You can always add more later.
  2. Pick your first three agents. For someone in Old Fourth Ward who’s retired, active in the community, and has a wife and dog, I’d suggest starting with: Local Guide, Culinary Advisor, and a Community/Activities Advisor.
  3. Use it for a week and correct it. When it gets something wrong or misses a preference, tell it. “Actually, we don’t eat shellfish” or “We prefer places on the east side of town.” These corrections become permanent memories that improve every future answer.

The goal isn’t perfection on day one. It’s building a tool that gets meaningfully better every week because it’s learning your life, not just the internet’s version of it.