Project Details

Browser-use review agent (Challenges 1, 3)

1. Designed a browsing loop (observe → reason → act) that reads the screenshot and page text together and decides where to look next
2. Set collection goals per review item (business details, terms, payment path) so the agent picks its next action from what is still missing
3. Made the agent stop on its own once it had what it needed, keeping captured screens and text as review evidence

Safety for third-party servers (Challenge 2)

1. Read public pages only, with one browser per merchant going one page at a time, so merchant servers see no more load than a single visitor

AI-driven improvement cycle (Challenge 1)

1. AI finds recurring format and evidence errors in run logs and review data, drafts prompt changes that fit the agent's structure, and re-runs the same data to compare
2. Errors a prompt cannot fix are shown to users as places where the data conflicts with the current direction (policy) or the policy has gaps, with supporting cases, affected scope and a proposed fix
3. Users set the direction and approve; AI does the editing, re-running and comparing, so the cycle is easy to repeat

AI features that cut the steps in between (Challenge 2)

1. AI gathers scattered material (chat threads, documents, sheets, owner notes), drafts the inspection requirements, and turns gaps into questions
2. From the requirements, AI proposes the components and their order and builds a draft inspection agent, flagging what it cannot build and what is missing
3. AI generates each task's prompts and supports versioned experiments, so prompt versions can be compared on results

Platform architecture and shared module (Challenges 2, 3)

1. Ran inspection agents on LangGraph; built the intermediate AI features (requirements, agent drafts, prompt learning, policy refinement) as Claude Agent SDK services
2. Kept policies, data, cases and input/output formats together as reference information, and connected each task's agent in a develop → run → improve flow
3. Stored each run's verdict, evidence, run history and version, and fed reviewers' final decisions (confirm, edit, hold) into the next improvement

Automatic guideline generation (Challenge 1)

1. Found the basis in the policy source text and turned a short policy into a guideline with decision criteria, violation/OK boundaries and policy references

Data-driven refinement (Challenges 2, 3)

1. Had AI compare the guideline's decisions with reviewers', find the patterns behind the disagreements and revise the guideline (e.g. marking compounds, amount units and ad-style phrasing common in ads as OK)
2. Split refinement and held-out data to confirm the improvement holds on unseen ads
3. People only checked the revisions; nobody edited the guideline by hand

Agent architecture with the Plan-and-Execute pattern and a Refine step (Challenge 1)

1. Plan: splits the request into sub-tasks and orders them
2. Execute (sequential): completes dependent sub-tasks in order
3. Execute (parallel): runs independent sub-tasks in parallel to cut latency
4. Refine: re-plans from the execution results to raise task accuracy

External and subscription calendar integration (Challenge 2)

1. Designed the flow so the LLM distinguishes external calendars (Outlook, Google Calendar) from subscription calendars (benefit and event schedules) and decides whether each is linked and can be read, changed or deleted

RAG for schedule retrieval (Challenge 1)

1. Filtering: narrows candidate events by owner, subscription and date range
2. Parallel search: splits candidates into N chunks and calls the LLM in parallel to find the requested events within the latency budget
3. Verification: checks that each extracted event matches the request and extracts the supporting evidence to improve accuracy

Prompt engineering for time information (Challenge 3)

1. Designed prompts that use the iCalendar standard and ISO 8601 durations so recurring events and reminders are handled precisely

Agent design for accuracy and UI/UX integration (Challenge 1)

1. Consolidated similar functions to simplify function selection
2. Let the conversation use in-app context such as the current playlist and UI screen
3. Split functions with low argument accuracy into sequential calls

Prompt engineering for cost and answer accuracy (Challenges 1, 2)

1. Kept prompts minimal to cut API cost and used fine-tuning to instill answer style
2. Switched argument extraction for hallucination-prone time functions to NER

Fine-tuning and evaluation (Challenge 3)

1. Defined the training-data format and generated multi-turn dialogue data
2. Fine-tuned GPT models on Azure OpenAI
3. Ran quantitative evaluation (function selection and accuracy, argument selection and extraction) and qualitative evaluation (multi-turn fluency, accuracy of function-result answers, handling of sensitive and inappropriate requests)

Slot-filling agent (Challenge 1)

1. Designed a function-calling LLM workflow that informs the user while collecting what the booking needs
2. Personalized recommendations using the user's location, recently visited cinemas and favorite theaters
3. Added a state-management module so slots collected in long conversations are not lost

RAG for movie search (Challenge 3)

1. Preprocessing: tags movie titles in the request with a Trie built from a title-synonym dictionary
2. Keyword extraction: uses the LLM to extract movie metadata and keywords from the request
3. Retrieval: vector search with metadata filtering on the extracted keywords
4. Final pick: uses the LLM to choose the movies that fit the request

Automated movie-metadata pipeline (Challenge 3)

1. Extracted keywords, summaries and metadata from daily movie updates with Airflow, an LLM and preprocessing modules
2. Converted the results to vectors and synced them to the vector DB

UI/UX and LLM workflow integration (Challenge 2)

1. Fed the results of UI actions (buttons, etc.) into the LLM workflow's prompt so both flow into each other naturally

Log-based data collection and training (Challenge 3)

1. Collected training data from service logs and fine-tuned GPT models on Azure OpenAI to raise function-calling accuracy and multi-turn fluency

Content-based recommendation pipeline (Challenge 1)

1. Classification and extraction: extracts location and category from the question (BERT-based classifiers and NER)
2. Filtering: narrows candidates by location, category and stop words
3. Encoding: masks place names and converts the question to a vector with the embedding model
4. Retrieval: finds similar questions in Elasticsearch with BM25 + cosine similarity
5. Re-ranking: scores question containment and question–answer consistency and returns the top N

Models behind the pipeline (Challenge 1)

1. Embedding model: trained a BERT-based model with a Siamese network so similar sentences sit close together
2. Hierarchical multi-class classifier: built hierarchical topic data and trained a BERT-based classifier
3. NER model: trained with BIO tagging to extract places and keywords

Model evaluation pipeline (Challenge 2)

1. Sent items whose category changed plus N sampled items to an external vendor for ground-truth labeling
2. Evaluated with F1 and accuracy and extracted items with low prediction accuracy
3. Gated deployment on a golden-set evaluation after retraining

Batch processing of new data (Challenge 1)

1. Automated an Airflow batch that encodes new question–answer pairs and re-indexes the vector DB so they become recommendation candidates