ScreenMates User Guide

Master AI-powered screenshot analysis with this comprehensive guide

📖 Table of Contents

🚀 Getting Started

Installation & First Launch

  1. Download ScreenMates from the Mac App Store
  2. Launch ScreenMates - it will appear in your menu bar (look for the viewfinder icon)
  3. Grant screen recording permission when prompted
  4. Select your preferred AI model from the settings
  5. Wait for the model to download and initialize

⚠️ Important Permission

ScreenMates requires screen recording permission to capture screenshots for analysis. This is essential for the app to function. All processing happens locally on your device - no data is sent to external servers.

System Requirements

  • macOS: 14.0 (Sonoma) or later
  • Hardware: Apple Silicon Mac (M1, M2, M3, or later)
  • Storage: 2-8 GB for AI models (varies by model)
  • Memory: 8 GB RAM recommended for optimal performance

Initial Setup

After installation, ScreenMates will guide you through the initial setup process:

  1. Click the ScreenMates icon in your menu bar
  2. Select "Settings" from the dropdown menu
  3. Choose your preferred Vision Language Model (VLM)
  4. Click "Load/Change VLM Model" to download and initialize
  5. Wait for the download to complete (this may take several minutes)

📸 Basic Screenshot Analysis

Taking Your First Screenshot

  1. Click the ScreenMates icon in your menu bar
  2. Select "Process Screenshot" from the menu
  3. Your cursor will change to a crosshair - drag to select the area you want to analyze
  4. Release to capture the selected area
  5. Wait for ScreenMates to process the image with AI
  6. View the analysis results in the popup window

🎯 Smart Area Selection

ScreenMates's screenshot tool allows you to select any rectangular area of your screen. This is perfect for capturing specific content like documents, charts, images, or text you want to analyze.

Understanding Analysis Results

ScreenMates's AI analysis can provide various types of insights:

  • Text Recognition: Extracts and identifies text in images
  • Content Description: Describes what's visible in the image
  • Data Extraction: Pulls out structured information from charts or tables
  • Context Understanding: Provides relevant insights about the content

Working with Results

Once you receive analysis results, you can:

  • Copy Text: Select and copy extracted text to your clipboard
  • Save Results: Keep the analysis for future reference
  • Process Again: Run additional analysis with different prompts

🔍 OCR & Text Extraction

Optimized OCR Mode

ScreenMates includes specialized OCR capabilities for accurate text extraction:

📝 Quick OCR

For pure text extraction, ScreenMates automatically optimizes its prompts to focus on accurate character recognition and text structure preservation.

Best Practices for OCR

  • High Contrast: Images with clear text on contrasting backgrounds work best
  • Proper Resolution: Ensure text is large enough to be clearly readable
  • Clean Capture: Avoid including unnecessary background elements
  • Language Support: Works best with English text, with good support for other Latin-based languages

Supported Content Types

  • Documents and PDFs
  • Screenshots of web pages
  • Handwritten text (with varying accuracy)
  • Text in images and graphics
  • Signs and printed materials
  • Code snippets and technical documentation

💡 Pro Tip

For the best OCR results, capture text at the highest resolution possible. If the original text is small, try zooming in before taking the screenshot.

✏️ Custom Prompts

Using the Custom Prompt Panel

  1. Click the ScreenMates icon in your menu bar
  2. Select "Custom Prompt" to open the prompt panel
  3. Enter your specific instruction or question
  4. Click "Take Screenshot & Process with This Prompt"
  5. Select the area you want to analyze
  6. Wait for the AI to process your request

Effective Prompt Writing

Write clear, specific prompts for better results:

✅ Good Prompt Examples

  • "Extract all the text from this document and format it cleanly"
  • "Summarize the key points from this chart"
  • "What programming language is this code written in?"
  • "List all the ingredients shown in this recipe"
  • "Translate this text to English"
  • "Explain what this error message means"

Prompt Categories

📄 Text Processing

  • Extract and clean up text
  • Summarize long content
  • Translate text to other languages
  • Format text in specific ways

📊 Data Analysis

  • Interpret charts and graphs
  • Extract data from tables
  • Analyze trends and patterns
  • Convert visual data to text format

🎨 Visual Analysis

  • Describe images and graphics
  • Identify objects and elements
  • Analyze design and layout
  • Explain visual concepts

💻 Technical Support

  • Explain error messages
  • Analyze code snippets
  • Debug configuration files
  • Interpret technical diagrams

💡 Advanced Tip

You can chain multiple requests by being specific about the output format. For example: "Extract the text from this image and format it as a bullet-point list" or "Analyze this chart and provide the data in CSV format."

⚙️ Settings & Configuration

Accessing Settings

  1. Click the ScreenMates icon in your menu bar
  2. Select "Settings" from the dropdown menu
  3. The settings window will open with various configuration options

VLM Model Selection

ScreenMates supports multiple Vision Language Models, each with different capabilities:

🤖 Model Comparison

  • Smaller Models: Faster processing, lower memory usage, good for basic OCR
  • Larger Models: More accurate analysis, better understanding, higher memory requirements
  • Specialized Models: Optimized for specific tasks like document analysis or code understanding

Auto-Start Configuration

Configure ScreenMates to automatically start when you log in:

  • Enable "Launch at Login" in settings
  • ScreenMates will appear in your menu bar automatically
  • The app runs quietly in the background until needed

Processing Parameters

Adjust how ScreenMates processes images:

  • Quality Settings: Balance between speed and accuracy
  • Default Prompts: Set preferred analysis modes
  • Memory Management: Configure model caching and memory usage

⚠️ Model Storage

AI models require significant storage space (2-8 GB each). Make sure you have adequate free space before downloading new models. You can delete unused models to free up space.

⌨️ Keyboard Shortcuts

Global Shortcuts

ScreenMates supports system-wide keyboard shortcuts for quick access:

🎹 Default Shortcuts

  • ⌘ + Shift + X: Global screenshot processing
  • ⌘ + Shift + P: Open custom prompt panel

Customizing Shortcuts

  1. Open ScreenMates Settings
  2. Navigate to the "Keyboard Shortcuts" section
  3. Click on the shortcut you want to change
  4. Press your desired key combination
  5. Save the changes

💡 Accessibility Tip

ScreenMates needs accessibility permissions to register global keyboard shortcuts. Grant this permission in System Settings > Privacy & Security > Accessibility when prompted.

🔧 Troubleshooting

Common Issues & Solutions

Screenshot Capture Not Working

🔍 Solution Steps

  1. Check screen recording permissions in System Settings > Privacy & Security
  2. Restart ScreenMates completely
  3. Verify that ScreenMates appears in the screen recording permissions list
  4. If missing, manually add ScreenMates to the allowed apps

AI Model Not Loading

  • Check internet connection - Models need to download initially
  • Verify storage space - Ensure you have enough free disk space
  • Restart the app - Close ScreenMates completely and relaunch
  • Clear model cache - Delete and re-download the model if corrupted

Poor OCR Results

  • Improve image quality - Ensure text is clear and high-contrast
  • Try different models - Some models perform better on specific content
  • Adjust capture area - Include less background noise in your screenshots
  • Use custom prompts - Be specific about what text you want extracted

Slow Performance

  • Close other apps - Free up system memory
  • Try a smaller model - Smaller models process faster
  • Check Activity Monitor - Ensure no other processes are consuming resources
  • Restart your Mac - Clear memory and refresh system resources

⚠️ If Problems Persist

If you continue to experience issues, try completely uninstalling and reinstalling ScreenMates. Make sure to back up any custom settings first. Contact support at [email protected] if problems continue.

💡 Tips & Best Practices

Maximizing Accuracy

  • Clean screenshots: Capture only the content you need to analyze
  • Good lighting: Ensure captured text and images are well-lit
  • Proper sizing: Don't capture text that's too small to read clearly
  • Stable capture: Avoid blurry or shaky screenshots

Workflow Optimization

🚀 Power User Tips

  • Batch processing: Prepare multiple screenshots and process them sequentially
  • Template prompts: Save commonly used custom prompts for quick reuse
  • Keyboard shortcuts: Learn and use shortcuts for faster access
  • Model switching: Use different models for different types of content

Creative Use Cases

  • Learning: Analyze educational content, diagrams, and textbooks
  • Research: Extract data from academic papers and reports
  • Productivity: Digitize handwritten notes and whiteboards
  • Accessibility: Read text from images for visually impaired users
  • Translation: Translate foreign text in real-time
  • Coding: Analyze code snippets and debug errors

Privacy & Security

🔒 Privacy Benefits

Remember that all ScreenMates processing happens locally on your device. Your screenshots never leave your Mac, ensuring complete privacy and security for sensitive information.

Performance Tips

  • Close unused apps: Free up memory for better AI performance
  • Regular restarts: Restart ScreenMates periodically to clear cache
  • Model management: Only keep the models you actively use
  • Monitor usage: Be aware of CPU and memory usage during processing