evan/aethera
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(chat): add optional photo upload support

Browse Source 
Add vision/multimodal support to chat, allowing users to send images
alongside or instead of text prompts. Images are transmitted and persisted
as base64 data URLs.

Backend:
- Add Images []string to Message struct for persistence
- Add Images []string to GenerateTextRequest with relaxed validation
- Build multimodal user messages using OpenAI SDK content parts
- Pass images through from handlers to client
- Deep-copy Images slice in message cloning

Frontend:
- Add images?: string[] to Message and GenerateTextRequest types
- Add image selection state and file input handler
- Add camera icon button, hidden file input, and image preview strip
- Render images in user message bubbles
- Pass images through to GenerateTextRequest

Tests:
- Add TestSendMessageWithImage for vision model testing
This commit is contained in:
Evan Reichard
2026-05-01 18:27:09 -04:00
parent c51c0ab070
commit e60b1ea8d5
9 changed files with 150 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

72
frontend/public/pages/chats.html
View File

@@ -14,6 +14,16 @@
: 'bg-primary-200 text-primary-900 rounded-bl-none'
]"
>
<!-- User Images -->
<div
x-show="message.role === 'user' && message.images && message.images.length > 0"
class="flex gap-1 mb-2 flex-wrap"
>
<template x-for="(img, imgIdx) in message.images" :key="imgIdx">
<img :src="img" class="max-w-full h-auto rounded-lg max-h-48" />
</template>
</div>
<!-- Thinking Section -->
<div
x-show="message.thinking"
@@ -161,8 +171,66 @@
</svg>
</div>
<!-- Image Preview Strip -->
<div
x-show="selectedImages.length > 0"
class="flex gap-2 flex-wrap"
>
<template x-for="(img, idx) in selectedImages" :key="idx">
<div class="relative">
<img :src="img" class="w-20 h-20 object-cover rounded-lg" />
<button
@click="selectedImages.splice(idx, 1)"
class="absolute -top-1 -right-1 w-5 h-5 bg-tertiary-700 text-white rounded-full flex items-center justify-center text-xs hover:bg-tertiary-900"
>
×
</button>
</div>
</template>
</div>
<!-- Message Form -->
<form @submit.prevent="sendMessage" class="flex gap-2 items-end">
<!-- Attach Image Button -->
<button
type="button"
@click="$refs.fileInput.click()"
class="self-stretch w-[44px] bg-primary-200 text-primary-700 rounded-xl transition-all flex items-center justify-center flex-shrink-0 hover:bg-primary-300 hover:shadow-md"
title="Attach Image"
>
<svg class="h-5 w-5" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
d="M3 9a2 2 0 012-2h.93a2 2 0 001.664-.89l.812-1.22A2 2 0 0110.07 4h3.86a2 2 0 011.664.89l.812 1.22A2 2 0 0018.07 7H19a2 2 0 012 2v9a2 2 0 01-2 2H5a2 2 0 01-2-2V9z"
/>
<path
stroke-linecap="round"
stroke-linejoin="round"
stroke-width="2"
d="M15 13a3 3 0 11-6 0 3 3 0 016 0z"
/>
</svg>
</button>
<!-- Hidden File Input -->
<input
x-ref="fileInput"
type="file"
accept="image/png, image/jpeg, image/webp"
multiple
class="hidden"
@change="
Array.from($event.target.files).forEach(file => {
const reader = new FileReader();
reader.onload = (e) => { selectedImages.push(e.target.result as string); };
reader.readAsDataURL(file);
});
$event.target.value = '';
"
/>
<textarea
x-model="inputMessage"
placeholder="Type your message..."
@@ -174,8 +242,8 @@
<button
type="submit"
:disabled="!inputMessage.trim() || loading"
:class="(!inputMessage.trim() || loading) ? 'opacity-50 cursor-not-allowed' : 'hover:shadow-md hover:scale-105'"
:disabled="(!inputMessage.trim() && selectedImages.length === 0) || loading"
:class=" ((!inputMessage.trim() && selectedImages.length === 0) || loading) ? 'opacity-50 cursor-not-allowed' : 'hover:shadow-md hover:scale-105'"
class="self-stretch w-[44px] bg-gradient-to-r from-primary-600 to-primary-500 text-white rounded-xl transition-all flex items-center justify-center flex-shrink-0"
>
<template x-if="loading">