feat(tools/looker): add support for pulse, vacuum and analyze audit and performance functions on a Looker instance (#1581)

This pull request adds 3 new tools, looker-health-pulse,
looker-health-vacuum, and looker-health-analyze, as capabilities to the
Looker MCP Toolbox. These tools are designed to provide health checks
and auditing analytical insights for a Looker instance (they come from
the popular [Looker CLI tool
Henry](https://github.com/looker-open-source/henry)).

**looker-health-pulse**
This tool performs various health checks on a Looker instance. It can be
used to:
- Check database connection status.
- Identify dashboards with slow-running or erroring queries.
- List slow explores and failed schedules.
- Find enabled legacy features.

**looker-health-analyze**
This tool performs analytical tasks on Looker projects, models, and
explores. It can be used to:
- Analyze projects to check Git status and validation.
- Analyze models to count explores and identify unused ones.
- Analyze explores to find unused joins and fields. *Unused is defined
as not being queried in the last 90 days.*

**looker-health-vacuum**
This tool finds unnused explores, joins, and fields based on user
defined search conditions (namely, timeframe and min query #):
- Identify unnused explores for specific or all models
- Identify unnused fields or joins for specific explores or all explores
within a model

This update targets Looker administrators, as it provides new
capabilities to monitor the health and efficiency of their Looker
instances and connect those capabilities to MCP Clients.

🛠️ Fixes #1415

---------

Co-authored-by: Luka Fontanilla <maluka@google.com>
This commit is contained in:
Dr. Strangelove
2025-10-02 13:03:45 -04:00
committed by GitHub
parent 3be9b7b3bd
commit 5aed4e136d
13 changed files with 2205 additions and 1 deletions

View File

@@ -106,6 +106,9 @@ import (
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookergetmeasures"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookergetmodels"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookergetparameters"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookerhealthanalyze"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookerhealthpulse"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookerhealthvacuum"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookermakedashboard"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookermakelook"
_ "github.com/googleapis/genai-toolbox/internal/tools/looker/lookerquery"

View File

@@ -1489,7 +1489,7 @@ func TestPrebuiltTools(t *testing.T) {
wantToolset: server.ToolsetConfigs{
"looker_tools": tools.ToolsetConfig{
Name: "looker_tools",
ToolNames: []string{"get_models", "get_explores", "get_dimensions", "get_measures", "get_filters", "get_parameters", "query", "query_sql", "query_url", "get_looks", "run_look", "make_look", "get_dashboards", "make_dashboard", "add_dashboard_element"},
ToolNames: []string{"get_models", "get_explores", "get_dimensions", "get_measures", "get_filters", "get_parameters", "query", "query_sql", "query_url", "get_looks", "run_look", "make_look", "get_dashboards", "make_dashboard", "add_dashboard_element", "health_pulse", "health_analyze", "health_vacuum"},
},
},
},

View File

@@ -0,0 +1,63 @@
---
title: "looker-health-analyze"
type: docs
weight: 1
description: >
"looker-health-analyze" provides a set of analytical commands for a Looker instance, allowing users to analyze projects, models, and explores.
aliases:
- /resources/tools/looker-health-analyze
---
## About
The `looker-health-analyze` tool performs various analysis tasks on a Looker instance. The `action` parameter selects the type of analysis to perform:
- `projects`: Analyzes all projects or a specified project, reporting on the number of models and view files, as well as Git connection and validation status.
- `models`: Analyzes all models or a specified model, providing a count of explores, unused explores, and total query counts.
- `explores`: Analyzes all explores or a specified explore, reporting on the number of joins, unused joins, fields, unused fields, and query counts. Being classified as **Unused** is determined by whether a field has been used as a field or filter within the past 90 days in production.
## Parameters
| **field** | **type** | **required** | **description** |
| :--- | :--- | :--- | :--- |
| kind | string | true | Must be "looker-health-analyze" |
| source | string | true | Looker source name |
| action | string | true | The analysis to perform: `projects`, `models`, or `explores`. |
| project | string | false | The name of the Looker project to analyze. |
| model | string | false | The name of the Looker model to analyze. Required for `explores` actions. |
| explore | string | false | The name of the Looker explore to analyze. Required for the `explores` action. |
| timeframe | int | false | The timeframe in days to analyze. Defaults to 90. |
| min_queries | int | false | The minimum number of queries for a model or explore to be considered used. Defaults to 1. |
## Example
Analyze all models in `thelook` project.
```yaml
tools:
analyze-tool:
kind: looker-health-analyze
source: looker-source
description: |
Analyzes Looker projects, models, and explores.
Specify the `action` parameter to select the type of analysis.
parameters:
action: models
project: "thelook"
Analyze all the explores in the `ecomm` model of `thelook` project. Specifically look at usage within the past 20 days. Usage minimum should be at least 10 queries.
```yaml
tools:
analyze-tool:
kind: looker-health-analyze
source: looker-source
description: |
Analyzes Looker projects, models, and explores.
Specify the `action` parameter to select the type of analysis.
parameters:
action: explores
project: "thelook"
model: "ecomm"
timeframe: 20
min_queries: 10

View File

@@ -0,0 +1,55 @@
---
title: "looker-health-pulse"
type: docs
weight: 1
description: >
"looker-health-pulse" performs health checks on a Looker instance, with multiple actions available (e.g., checking database connections, dashboard performance, etc).
aliases:
- /resources/tools/looker-health-pulse
---
## About
The `looker-health-pulse` tool performs health checks on a Looker instance. The `action` parameter selects the type of check to perform:
- `check_db_connections`: Checks all database connections, runs supported tests, and reports query counts.
- `check_dashboard_performance`: Finds dashboards with slow running queries in the last 7 days.
- `check_dashboard_errors`: Lists dashboards with erroring queries in the last 7 days.
- `check_explore_performance`: Lists the slowest explores in the last 7 days and reports average query runtime.
- `check_schedule_failures`: Lists schedules that have failed in the last 7 days.
- `check_legacy_features`: Lists enabled legacy features. (*To note, this function is not available in Looker Core. You will get an error running this command with a Core instance configured.*)
## Parameters
| **field** | **type** | **required** | **description** |
|---------------|:--------:|:------------:|---------------------------------------------|
| kind | string | true | Must be "looker-health-pulse" |
| source | string | true | Looker source name |
| action | string | true | The health check to perform |
## Example
```yaml
tools:
pulse:
kind: looker-health-pulse
source: looker-source
description: |
Pulse Tool
Performs health checks on Looker instance.
Specify the `action` parameter to select the check.
parameters:
action: check_dashboard_performance
```
## Reference
| **action** | **description** |
|---------------------------|--------------------------------------------------------------------------------|
| check_db_connections | Checks all database connections and reports query counts and errors |
| check_dashboard_performance | Finds dashboards with slow queries (>30s) in the last 7 days |
| check_dashboard_errors | Lists dashboards with erroring queries in the last 7 days |
| check_explore_performance | Lists slowest explores and average query runtime |
| check_schedule_failures | Lists failed schedules in the last 7 days |
| check_legacy_features | Lists enabled legacy features |

View File

@@ -0,0 +1,63 @@
---
title: "looker-health-vacuum"
type: docs
weight: 1
description: >
"looker-health-vacuum" provides a set of commands to audit and identify unused LookML objects in a Looker instance.
aliases:
- /resources/tools/looker-health-vacuum
---
## About
The `looker-health-vacuum` tool helps you identify unused LookML objects such as models, explores, joins, and fields. The `action` parameter selects the type of vacuum to perform:
- `models`: Identifies unused explores within a model.
- `explores`: Identifies unused joins and fields within an explore.
## Parameters
| **field** | **type** | **required** | **description** |
| :--- | :--- | :--- | :--- |
| kind | string | true | Must be "looker-health-vacuum" |
| source | string | true | Looker source name |
| action | string | true | The vacuum to perform: `models`, or `explores`. |
| project | string | false | The name of the Looker project to vacuum. |
| model | string | false | The name of the Looker model to vacuum. |
| explore | string | false | The name of the Looker explore to vacuum. |
| timeframe | int | false | The timeframe in days to analyze for usage. Defaults to 90. |
| min_queries | int | false | The minimum number of queries for an object to be considered used. Defaults to 1. |
## Example
Identify unnused fields (*in this case, less than 1 query in the last 20 days*) and joins in the `order_items` explore and `thelook` model
```yaml
tools:
vacuum-tool:
kind: looker-health-vacuum
source: looker-source
description: |
Vacuums the Looker instance by identifying unused explores, fields, and joins.
parameters:
action: explores
project: "thelook_core"
model: "thelook"
explore: "order_items"
timeframe: 20
min_queries: 1
```
Identify unnused explores across all models in `thelook_core` project.
```yaml
tools:
vacuum-tool:
kind: looker-health-vacuum
source: looker-source
description: |
Vacuums the Looker instance by identifying unused explores, fields, and joins.
parameters:
action: models
project: "thelook_core"

View File

@@ -695,6 +695,56 @@ tools:
This tool can be called many times for one dashboard_id
and the resulting tiles will be added in order.
health_pulse:
kind: looker-health-pulse
source: looker-source
description: |
health-pulse Tool
This tool takes the pulse of a Looker instance by taking
one of the following actions:
1. `check_db_connections`,
2. `check_dashboard_performance`,
3. `check_dashboard_errors`,
4. `check_explore_performance`,
5. `check_schedule_failures`, or
6. `check_legacy_features`
health_analyze:
kind: looker-health-analyze
source: looker-source
description: |
health-analyze Tool
This tool calculates the usage of projects, models and explores.
It accepts 6 parameters:
1. `action`: can be "projects", "models", or "explores"
2. `project`: the project to analyze (optional)
3. `model`: the model to analyze (optional)
4. `explore`: the explore to analyze (optional)
5. `timeframe`: the lookback period in days, default is 90
6. `min_queries`: the minimum number of queries to consider a resource as active, default is 1
health_vacuum:
kind: looker-health-vacuum
source: looker-source
description: |
health-vacuum Tool
This tool suggests models or explores that can removed
because they are unused.
It accepts 6 parameters:
1. `action`: can be "models" or "explores"
2. `project`: the project to vacuum (optional)
3. `model`: the model to vacuum (optional)
4. `explore`: the explore to vacuum (optional)
5. `timeframe`: the lookback period in days, default is 90
6. `min_queries`: the minimum number of queries to consider a resource as active, default is 1
The result is a list of objects that are candidates for deletion.
toolsets:
looker_tools:
- get_models
@@ -712,3 +762,6 @@ toolsets:
- get_dashboards
- make_dashboard
- add_dashboard_element
- health_pulse
- health_analyze
- health_vacuum

View File

@@ -0,0 +1,555 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package lookerhealthanalyze
import (
"context"
"encoding/json"
"fmt"
"regexp"
"strings"
yaml "github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
lookersrc "github.com/googleapis/genai-toolbox/internal/sources/looker"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/tools/looker/lookercommon"
"github.com/googleapis/genai-toolbox/internal/util"
"github.com/looker-open-source/sdk-codegen/go/rtl"
v4 "github.com/looker-open-source/sdk-codegen/go/sdk/v4"
)
// =================================================================================================================
// START MCP SERVER CORE LOGIC
// =================================================================================================================
const kind string = "looker-health-analyze"
func init() {
if !tools.Register(kind, newConfig) {
panic(fmt.Sprintf("tool kind %q already registered", kind))
}
}
func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.ToolConfig, error) {
actual := Config{Name: name}
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
return actual, nil
}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
Source string `yaml:"source" validate:"required"`
Description string `yaml:"description" validate:"required"`
AuthRequired []string `yaml:"authRequired"`
Parameters map[string]any `yaml:"parameters"`
}
var _ tools.ToolConfig = Config{}
func (cfg Config) ToolConfigKind() string {
return kind
}
func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error) {
rawS, ok := srcs[cfg.Source]
if !ok {
return nil, fmt.Errorf("no source named %q configured", cfg.Source)
}
s, ok := rawS.(*lookersrc.Source)
if !ok {
return nil, fmt.Errorf("invalid source for %q tool: source kind must be `looker`", kind)
}
actionParameter := tools.NewStringParameterWithRequired("action", "The analysis to run. Can be 'projects', 'models', or 'explores'.", true)
projectParameter := tools.NewStringParameterWithRequired("project", "The Looker project to analyze (optional).", false)
modelParameter := tools.NewStringParameterWithRequired("model", "The Looker model to analyze (optional).", false)
exploreParameter := tools.NewStringParameterWithRequired("explore", "The Looker explore to analyze (optional).", false)
timeframeParameter := tools.NewIntParameterWithDefault("timeframe", 90, "The timeframe in days to analyze.")
minQueriesParameter := tools.NewIntParameterWithDefault("min_queries", 0, "The minimum number of queries for a model or explore to be considered used.")
parameters := tools.Parameters{
actionParameter,
projectParameter,
modelParameter,
exploreParameter,
timeframeParameter,
minQueriesParameter,
}
mcpManifest := tools.GetMcpManifest(cfg.Name, cfg.Description, cfg.AuthRequired, parameters)
return Tool{
Name: cfg.Name,
Kind: kind,
Parameters: parameters,
AuthRequired: cfg.AuthRequired,
UseClientOAuth: s.UseClientOAuth,
Client: s.Client,
ApiSettings: s.ApiSettings,
manifest: tools.Manifest{
Description: cfg.Description,
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
}, nil
}
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
UseClientOAuth bool
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters
manifest tools.Manifest
mcpManifest tools.McpManifest
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
sdk, err := lookercommon.GetLookerSDK(t.UseClientOAuth, t.ApiSettings, t.Client, accessToken)
if err != nil {
return nil, fmt.Errorf("error getting sdk: %w", err)
}
paramsMap := params.AsMap()
timeframe, _ := paramsMap["timeframe"].(int)
if timeframe == 0 {
timeframe = 90
}
minQueries, _ := paramsMap["min_queries"].(int)
if minQueries == 0 {
minQueries = 1
}
analyzeTool := &analyzeTool{
SdkClient: sdk,
timeframe: timeframe,
minQueries: minQueries,
}
action, ok := paramsMap["action"].(string)
if !ok {
return nil, fmt.Errorf("action parameter not found")
}
switch action {
case "projects":
projectId, _ := paramsMap["project"].(string)
result, err := analyzeTool.projects(ctx, projectId)
if err != nil {
return nil, fmt.Errorf("error analyzing projects: %w", err)
}
logger.DebugContext(ctx, "result = ", result)
return result, nil
case "models":
projectName, _ := paramsMap["project"].(string)
modelName, _ := paramsMap["model"].(string)
result, err := analyzeTool.models(ctx, projectName, modelName)
if err != nil {
return nil, fmt.Errorf("error analyzing models: %w", err)
}
logger.DebugContext(ctx, "result = ", result)
return result, nil
case "explores":
modelName, _ := paramsMap["model"].(string)
exploreName, _ := paramsMap["explore"].(string)
result, err := analyzeTool.explores(ctx, modelName, exploreName)
if err != nil {
return nil, fmt.Errorf("error analyzing explores: %w", err)
}
logger.DebugContext(ctx, "result = ", result)
return result, nil
default:
return nil, fmt.Errorf("unknown action: %s", action)
}
}
func (t Tool) ParseParams(data map[string]any, claims map[string]map[string]any) (tools.ParamValues, error) {
return tools.ParseParams(t.Parameters, data, claims)
}
func (t Tool) Manifest() tools.Manifest {
return t.manifest
}
func (t Tool) McpManifest() tools.McpManifest {
return t.mcpManifest
}
func (t Tool) Authorized(verifiedAuthServices []string) bool {
return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices)
}
func (t Tool) RequiresClientAuthorization() bool {
return t.UseClientOAuth
}
// =================================================================================================================
// END MCP SERVER CORE LOGIC
// =================================================================================================================
// =================================================================================================================
// START LOOKER HEALTH ANALYZE CORE LOGIC
// =================================================================================================================
type analyzeTool struct {
SdkClient *v4.LookerSDK
timeframe int
minQueries int
}
func (t *analyzeTool) projects(ctx context.Context, id string) ([]map[string]interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
var projects []*v4.Project
if id != "" {
p, err := t.SdkClient.Project(id, "", nil)
if err != nil {
return nil, fmt.Errorf("error fetching project %s: %w", id, err)
}
projects = append(projects, &p)
} else {
allProjects, err := t.SdkClient.AllProjects("", nil)
if err != nil {
return nil, fmt.Errorf("error fetching all projects: %w", err)
}
for i := range allProjects {
projects = append(projects, &allProjects[i])
}
}
var results []map[string]interface{}
for _, p := range projects {
pName := *p.Name
pID := *p.Id
logger.InfoContext(ctx, fmt.Sprintf("Analyzing project: %s", pName))
projectFiles, err := t.SdkClient.AllProjectFiles(pID, "", nil)
if err != nil {
return nil, fmt.Errorf("error fetching files for project %s: %w", pName, err)
}
modelCount := 0
viewFileCount := 0
for _, f := range projectFiles {
if f.Type != nil {
if *f.Type == "model" {
modelCount++
}
if *f.Type == "view" {
viewFileCount++
}
}
}
gitConnectionStatus := "OK"
if p.GitRemoteUrl == nil {
gitConnectionStatus = "No repo found"
} else if strings.Contains(*p.GitRemoteUrl, "/bare_models/") {
gitConnectionStatus = "Bare repo, no tests required"
}
results = append(results, map[string]interface{}{
"Project": pName,
"# Models": modelCount,
"# View Files": viewFileCount,
"Git Connection Status": gitConnectionStatus,
"PR Mode": string(*p.PullRequestMode),
"Is Validation Required": *p.ValidationRequired,
})
}
return results, nil
}
func (t *analyzeTool) models(ctx context.Context, project, model string) ([]map[string]interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Analyzing models...")
usedModels, err := t.getUsedModels(ctx)
if err != nil {
return nil, err
}
lookmlModels, err := t.SdkClient.AllLookmlModels(v4.RequestAllLookmlModels{}, nil)
if err != nil {
return nil, fmt.Errorf("error fetching LookML models: %w", err)
}
var results []map[string]interface{}
for _, m := range lookmlModels {
if (project == "" || (m.ProjectName != nil && *m.ProjectName == project)) &&
(model == "" || (m.Name != nil && *m.Name == model)) {
queryCount := 0
if qc, ok := usedModels[*m.Name]; ok {
queryCount = qc
}
exploreCount := 0
if m.Explores != nil {
exploreCount = len(*m.Explores)
}
results = append(results, map[string]interface{}{
"Project": *m.ProjectName,
"Model": *m.Name,
"# Explores": exploreCount,
"Query Count": queryCount,
})
}
}
return results, nil
}
func (t *analyzeTool) getUsedModels(ctx context.Context) (map[string]int, error) {
limit := "5000"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"history.query_run_count", "query.model"},
Filters: &map[string]any{
"history.created_date": fmt.Sprintf("%d days", t.timeframe),
"query.model": "-system__activity, -i__looker",
"history.query_run_count": fmt.Sprintf(">%d", t.minQueries-1),
"user.dev_branch_name": "NULL",
},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", nil)
if err != nil {
return nil, err
}
var data []map[string]interface{}
_ = json.Unmarshal([]byte(raw), &data)
results := make(map[string]int)
for _, row := range data {
model, _ := row["query.model"].(string)
count, _ := row["history.query_run_count"].(float64)
results[model] = int(count)
}
return results, nil
}
func (t *analyzeTool) getUsedExploreFields(ctx context.Context, model, explore string) (map[string]int, error) {
limit := "5000"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"query.formatted_fields", "query.filters", "history.query_run_count"},
Filters: &map[string]any{
"history.created_date": fmt.Sprintf("%d days", t.timeframe),
"query.model": strings.ReplaceAll(model, "_", "^_"),
"query.view": strings.ReplaceAll(explore, "_", "^_"),
"query.formatted_fields": "-NULL",
"history.workspace_id": "production",
},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", nil)
if err != nil {
return nil, err
}
var data []map[string]interface{}
_ = json.Unmarshal([]byte(raw), &data)
results := make(map[string]int)
fieldRegex := regexp.MustCompile(`(\w+\.\w+)`)
for _, row := range data {
count, _ := row["history.query_run_count"].(float64)
formattedFields, _ := row["query.formatted_fields"].(string)
filters, _ := row["query.filters"].(string)
usedFields := make(map[string]bool)
for _, field := range fieldRegex.FindAllString(formattedFields, -1) {
results[field] += int(count)
usedFields[field] = true
}
for _, field := range fieldRegex.FindAllString(filters, -1) {
if _, ok := usedFields[field]; !ok {
results[field] += int(count)
}
}
}
return results, nil
}
func (t *analyzeTool) explores(ctx context.Context, model, explore string) ([]map[string]interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Analyzing explores...")
lookmlModels, err := t.SdkClient.AllLookmlModels(v4.RequestAllLookmlModels{}, nil)
if err != nil {
return nil, fmt.Errorf("error fetching LookML models: %w", err)
}
var results []map[string]interface{}
for _, m := range lookmlModels {
if model != "" && (m.Name == nil || *m.Name != model) {
continue
}
if m.Explores == nil {
continue
}
for _, e := range *m.Explores {
if explore != "" && (e.Name == nil || *e.Name != explore) {
continue
}
if e.Name == nil {
continue
}
// Get detailed explore info to count fields and joins
req := v4.RequestLookmlModelExplore{
LookmlModelName: *m.Name,
ExploreName: *e.Name,
}
exploreDetail, err := t.SdkClient.LookmlModelExplore(req, nil)
if err != nil {
// Log the error but continue to the next explore if possible
logger.ErrorContext(ctx, fmt.Sprintf("Error fetching detail for explore %s.%s: %v", *m.Name, *e.Name, err))
continue
}
fieldCount := 0
if exploreDetail.Fields != nil {
fieldCount = len(*exploreDetail.Fields.Dimensions) + len(*exploreDetail.Fields.Measures)
}
joinCount := 0
if exploreDetail.Joins != nil {
joinCount = len(*exploreDetail.Joins)
}
usedFields, err := t.getUsedExploreFields(ctx, *m.Name, *e.Name)
if err != nil {
logger.ErrorContext(ctx, fmt.Sprintf("Error fetching used fields for explore %s.%s: %v", *m.Name, *e.Name, err))
continue
}
allFields := []string{}
if exploreDetail.Fields != nil {
for _, d := range *exploreDetail.Fields.Dimensions {
if !*d.Hidden {
allFields = append(allFields, *d.Name)
}
}
for _, ms := range *exploreDetail.Fields.Measures {
if !*ms.Hidden {
allFields = append(allFields, *ms.Name)
}
}
}
unusedFieldsCount := 0
for _, field := range allFields {
if _, ok := usedFields[field]; !ok {
unusedFieldsCount++
}
}
joinStats := make(map[string]int)
if exploreDetail.Joins != nil {
for field, queryCount := range usedFields {
join := strings.Split(field, ".")[0]
joinStats[join] += queryCount
}
for _, join := range *exploreDetail.Joins {
if _, ok := joinStats[*join.Name]; !ok {
joinStats[*join.Name] = 0
}
}
}
unusedJoinsCount := 0
for _, count := range joinStats {
if count == 0 {
unusedJoinsCount++
}
}
// Use an inline query to get query count for the explore
limit := "1"
queryCountQueryBody := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"history.query_run_count"},
Filters: &map[string]any{
"query.model": *m.Name,
"query.view": *e.Name,
"history.created_date": fmt.Sprintf("%d days", t.timeframe),
"history.query_run_count": fmt.Sprintf(">%d", t.minQueries-1),
"user.dev_branch_name": "NULL",
},
Limit: &limit,
}
rawQueryCount, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, queryCountQueryBody, "json", nil)
if err != nil {
return nil, err
}
queryCount := 0
var data []map[string]interface{}
_ = json.Unmarshal([]byte(rawQueryCount), &data)
if len(data) > 0 {
if count, ok := data[0]["history.query_run_count"].(float64); ok {
queryCount = int(count)
}
}
results = append(results, map[string]interface{}{
"Model": *m.Name,
"Explore": *e.Name,
"Is Hidden": *e.Hidden,
"Has Description": e.Description != nil && *e.Description != "",
"# Joins": joinCount,
"# Unused Joins": unusedJoinsCount,
"# Unused Fields": unusedFieldsCount,
"# Fields": fieldCount,
"Query Count": queryCount,
})
}
}
return results, nil
}
// =================================================================================================================
// END LOOKER HEALTH ANALYZE CORE LOGIC
// =================================================================================================================

View File

@@ -0,0 +1,113 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package lookerhealthanalyze_test
import (
"strings"
"testing"
yaml "github.com/goccy/go-yaml"
"github.com/google/go-cmp/cmp"
"github.com/googleapis/genai-toolbox/internal/server"
"github.com/googleapis/genai-toolbox/internal/testutils"
lha "github.com/googleapis/genai-toolbox/internal/tools/looker/lookerhealthanalyze"
)
func TestParseFromYamlLookerHealthAnalyze(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
want server.ToolConfigs
}{
{
desc: "basic example",
in: `
tools:
example_tool:
kind: looker-health-analyze
source: my-instance
description: some description
`,
want: server.ToolConfigs{
"example_tool": lha.Config{
Name: "example_tool",
Kind: "looker-health-analyze",
Source: "my-instance",
Description: "some description",
AuthRequired: []string{},
},
},
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
// Parse contents
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err != nil {
t.Fatalf("unable to unmarshal: %s", err)
}
if diff := cmp.Diff(tc.want, got.Tools); diff != "" {
t.Fatalf("incorrect parse: diff %v", diff)
}
})
}
}
func TestFailParseFromYamlLookerHealthAnalyze(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
err string
}{
{
desc: "Invalid field",
in: `
tools:
example_tool:
kind: looker-health-analyze
source: my-instance
invalid_field: true
`,
err: "unable to parse tool \"example_tool\" as kind \"looker-health-analyze\": [2:1] unknown field \"invalid_field\"",
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
// Parse contents
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err == nil {
t.Fatalf("expect parsing to fail")
}
errStr := err.Error()
if !strings.Contains(errStr, tc.err) {
t.Fatalf("unexpected error string: got %q, want substring %q", errStr, tc.err)
}
})
}
}

View File

@@ -0,0 +1,459 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package lookerhealthpulse
import (
"context"
"encoding/json"
"fmt"
"strings"
yaml "github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
lookersrc "github.com/googleapis/genai-toolbox/internal/sources/looker"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/tools/looker/lookercommon"
"github.com/googleapis/genai-toolbox/internal/util"
"github.com/looker-open-source/sdk-codegen/go/rtl"
v4 "github.com/looker-open-source/sdk-codegen/go/sdk/v4"
)
// =================================================================================================================
// START MCP SERVER CORE LOGIC
// =================================================================================================================
const kind string = "looker-health-pulse"
func init() {
if !tools.Register(kind, newConfig) {
panic(fmt.Sprintf("tool kind %q already registered", kind))
}
}
func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.ToolConfig, error) {
actual := Config{Name: name}
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
return actual, nil
}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
Source string `yaml:"source" validate:"required"`
Description string `yaml:"description" validate:"required"`
AuthRequired []string `yaml:"authRequired"`
Parameters map[string]any `yaml:"parameters"`
}
// validate interface
var _ tools.ToolConfig = Config{}
func (cfg Config) ToolConfigKind() string {
return kind
}
func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error) {
// verify source exists
rawS, ok := srcs[cfg.Source]
if !ok {
return nil, fmt.Errorf("no source named %q configured", cfg.Source)
}
// verify the source is compatible
s, ok := rawS.(*lookersrc.Source)
if !ok {
return nil, fmt.Errorf("invalid source for %q tool: source kind must be `looker`", kind)
}
actionParameter := tools.NewStringParameterWithRequired("action", "The health check to run. Can be either: `check_db_connections`, `check_dashboard_performance`,`check_dashboard_errors`,`check_explore_performance`,`check_schedule_failures`, or `check_legacy_features`", true)
parameters := tools.Parameters{
actionParameter,
}
mcpManifest := tools.GetMcpManifest(cfg.Name, cfg.Description, cfg.AuthRequired, parameters)
// finish tool setup
return Tool{
Name: cfg.Name,
Kind: kind,
Parameters: parameters,
AuthRequired: cfg.AuthRequired,
UseClientOAuth: s.UseClientOAuth,
Client: s.Client,
ApiSettings: s.ApiSettings,
manifest: tools.Manifest{
Description: cfg.Description,
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
}, nil
}
// validate interface
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
UseClientOAuth bool
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters `yaml:"parameters"`
manifest tools.Manifest
mcpManifest tools.McpManifest
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
sdk, err := lookercommon.GetLookerSDK(t.UseClientOAuth, t.ApiSettings, t.Client, accessToken)
if err != nil {
return nil, fmt.Errorf("error getting sdk: %w", err)
}
pulseTool := &pulseTool{
ApiSettings: t.ApiSettings,
SdkClient: sdk,
}
paramsMap := params.AsMap()
action, ok := paramsMap["action"].(string)
if !ok {
return nil, fmt.Errorf("action parameter not found")
}
pulseParams := PulseParams{
Action: action,
}
result, err := pulseTool.RunPulse(ctx, pulseParams)
if err != nil {
return nil, fmt.Errorf("error running pulse: %w", err)
}
logger.DebugContext(ctx, "result = ", result)
return result, nil
}
func (t Tool) ParseParams(data map[string]any, claims map[string]map[string]any) (tools.ParamValues, error) {
return tools.ParseParams(t.Parameters, data, claims)
}
func (t Tool) Manifest() tools.Manifest {
return t.manifest
}
func (t Tool) McpManifest() tools.McpManifest {
return t.mcpManifest
}
func (t Tool) Authorized(verifiedAuthServices []string) bool {
return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices)
}
func (t Tool) RequiresClientAuthorization() bool {
return t.UseClientOAuth
}
// =================================================================================================================
// END MCP SERVER CORE LOGIC
// =================================================================================================================
// =================================================================================================================
// START LOOKER HEALTH PULSE CORE LOGIC
// =================================================================================================================
type PulseParams struct {
Action string
// Optionally add more parameters if needed
}
// pulseTool holds Looker API settings and client
type pulseTool struct {
ApiSettings *rtl.ApiSettings
SdkClient *v4.LookerSDK
}
func (t *pulseTool) RunPulse(ctx context.Context, params PulseParams) (interface{}, error) {
switch params.Action {
case "check_db_connections":
return t.checkDBConnections(ctx)
case "check_dashboard_performance":
return t.checkDashboardPerformance(ctx)
case "check_dashboard_errors":
return t.checkDashboardErrors(ctx)
case "check_explore_performance":
return t.checkExplorePerformance(ctx)
case "check_schedule_failures":
return t.checkScheduleFailures(ctx)
case "check_legacy_features":
return t.checkLegacyFeatures(ctx)
default:
return nil, fmt.Errorf("unknown action: %s", params.Action)
}
}
// Check DB connections and run tests
func (t *pulseTool) checkDBConnections(ctx context.Context) (interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Test 1/6: Checking connections")
reservedNames := map[string]struct{}{
"looker__internal__analytics__replica": {},
"looker__internal__analytics": {},
"looker": {},
"looker__ilooker": {},
}
connections, err := t.SdkClient.AllConnections("", t.ApiSettings)
if err != nil {
return nil, fmt.Errorf("error fetching connections: %w", err)
}
var filteredConnections []v4.DBConnection
for _, c := range connections {
if _, reserved := reservedNames[*c.Name]; !reserved {
filteredConnections = append(filteredConnections, c)
}
}
if len(filteredConnections) == 0 {
return nil, fmt.Errorf("no connections found")
}
var results []map[string]interface{}
for _, conn := range filteredConnections {
var errors []string
// Test connection (simulate test_connection endpoint)
resp, err := t.SdkClient.TestConnection(*conn.Name, nil, t.ApiSettings)
if err != nil {
errors = append(errors, "API JSONDecode Error")
} else {
for _, r := range resp {
if *r.Status == "error" {
errors = append(errors, *r.Message)
}
}
}
// Run inline query for connection activity
limit := "1"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"history.query_run_count"},
Filters: &map[string]any{
"history.connection_name": *conn.Name,
"history.created_date": "90 days",
"user.dev_branch_name": "NULL",
},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", t.ApiSettings)
if err != nil {
return nil, err
}
var queryRunCount interface{}
var data []map[string]interface{}
_ = json.Unmarshal([]byte(raw), &data)
if len(data) > 0 {
queryRunCount = data[0]["history.query_run_count"]
}
results = append(results, map[string]interface{}{
"Connection": *conn.Name,
"Status": "OK",
"Errors": errors,
"Query Count": queryRunCount,
})
}
return results, nil
}
func (t *pulseTool) checkDashboardPerformance(ctx context.Context) (interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Test 2/6: Checking for dashboards with queries slower than 30 seconds in the last 7 days")
limit := "20"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"dashboard.title", "query.count"},
Filters: &map[string]any{
"history.created_date": "7 days",
"history.real_dash_id": "-NULL",
"history.runtime": ">30",
"history.status": "complete",
},
Sorts: &[]string{"query.count desc"},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", t.ApiSettings)
if err != nil {
return nil, err
}
var dashboards []map[string]interface{}
if err := json.Unmarshal([]byte(raw), &dashboards); err != nil {
return nil, err
}
return dashboards, nil
}
func (t *pulseTool) checkDashboardErrors(ctx context.Context) (interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Test 3/6: Checking for dashboards with erroring queries in the last 7 days")
limit := "20"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"dashboard.title", "history.query_run_count"},
Filters: &map[string]any{
"dashboard.title": "-NULL",
"history.created_date": "7 days",
"history.dashboard_session": "-NULL",
"history.status": "error",
},
Sorts: &[]string{"history.query_run_count desc"},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", t.ApiSettings)
if err != nil {
return nil, err
}
var dashboards []map[string]interface{}
if err := json.Unmarshal([]byte(raw), &dashboards); err != nil {
return nil, err
}
return dashboards, nil
}
func (t *pulseTool) checkExplorePerformance(ctx context.Context) (interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Test 4/6: Checking for the slowest explores in the past 7 days")
limit := "20"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"query.model", "query.view", "history.average_runtime"},
Filters: &map[string]any{
"history.created_date": "7 days",
"query.model": "-NULL, -system^_^_activity",
},
Sorts: &[]string{"history.average_runtime desc"},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", t.ApiSettings)
if err != nil {
return nil, err
}
var explores []map[string]interface{}
if err := json.Unmarshal([]byte(raw), &explores); err != nil {
return nil, err
}
// Average query runtime
query.Fields = &[]string{"history.average_runtime"}
rawAvg, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", t.ApiSettings)
if err != nil {
return nil, err
}
var avgData []map[string]interface{}
if err := json.Unmarshal([]byte(rawAvg), &avgData); err == nil {
if len(avgData) > 0 {
if avgRuntime, ok := avgData[0]["history.average_runtime"].(float64); ok {
logger.InfoContext(ctx, fmt.Sprintf("For context, the average query runtime is %.4fs", avgRuntime))
}
}
}
return explores, nil
}
func (t *pulseTool) checkScheduleFailures(ctx context.Context) (interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Test 5/6: Checking for failing schedules")
limit := "500"
query := &v4.WriteQuery{
Model: "system__activity",
View: "scheduled_plan",
Fields: &[]string{"scheduled_job.name", "scheduled_job.count"},
Filters: &map[string]any{
"scheduled_job.created_date": "7 days",
"scheduled_job.status": "failure",
},
Sorts: &[]string{"scheduled_job.count desc"},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", t.ApiSettings)
if err != nil {
return nil, err
}
var schedules []map[string]interface{}
if err := json.Unmarshal([]byte(raw), &schedules); err != nil {
return nil, err
}
return schedules, nil
}
func (t *pulseTool) checkLegacyFeatures(ctx context.Context) (interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Test 6/6: Checking for enabled legacy features")
features, err := t.SdkClient.AllLegacyFeatures(t.ApiSettings)
if err != nil {
if strings.Contains(err.Error(), "Unsupported in Looker (Google Cloud core)") {
return []map[string]string{{"Feature": "Unsupported in Looker (Google Cloud core)"}}, nil
}
logger.ErrorContext(ctx, err.Error())
return []map[string]string{{"Feature": "Unable to pull legacy features due to SDK error"}}, nil
}
var legacyFeatures []map[string]string
for _, f := range features {
if *f.Enabled {
legacyFeatures = append(legacyFeatures, map[string]string{"Feature": *f.Name})
}
}
return legacyFeatures, nil
}
// =================================================================================================================
// END LOOKER HEALTH PULSE CORE LOGIC
// =================================================================================================================

View File

@@ -0,0 +1,113 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package lookerhealthpulse_test
import (
"strings"
"testing"
yaml "github.com/goccy/go-yaml"
"github.com/google/go-cmp/cmp"
"github.com/googleapis/genai-toolbox/internal/server"
"github.com/googleapis/genai-toolbox/internal/testutils"
lhp "github.com/googleapis/genai-toolbox/internal/tools/looker/lookerhealthpulse"
)
func TestParseFromYamlLookerHealthPulse(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
want server.ToolConfigs
}{
{
desc: "basic example",
in: `
tools:
example_tool:
kind: looker-health-pulse
source: my-instance
description: some description
`,
want: server.ToolConfigs{
"example_tool": lhp.Config{
Name: "example_tool",
Kind: "looker-health-pulse",
Source: "my-instance",
Description: "some description",
AuthRequired: []string{},
},
},
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
// Parse contents
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err != nil {
t.Fatalf("unable to unmarshal: %s", err)
}
if diff := cmp.Diff(tc.want, got.Tools); diff != "" {
t.Fatalf("incorrect parse: diff %v", diff)
}
})
}
}
func TestFailParseFromYamlLookerHealthPulse(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
err string
}{
{
desc: "Invalid field",
in: `
tools:
example_tool:
kind: looker-health-pulse
source: my-instance
invalid_field: true
`,
err: "unable to parse tool \"example_tool\" as kind \"looker-health-pulse\": [2:1] unknown field \"invalid_field\"",
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
// Parse contents
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err == nil {
t.Fatalf("expect parsing to fail")
}
errStr := err.Error()
if !strings.Contains(errStr, tc.err) {
t.Fatalf("unexpected error string: got %q, want substring %q", errStr, tc.err)
}
})
}
}

View File

@@ -0,0 +1,459 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package lookerhealthvacuum
import (
"context"
"encoding/json"
"fmt"
"regexp"
"strings"
yaml "github.com/goccy/go-yaml"
"github.com/googleapis/genai-toolbox/internal/sources"
lookersrc "github.com/googleapis/genai-toolbox/internal/sources/looker"
"github.com/googleapis/genai-toolbox/internal/tools"
"github.com/googleapis/genai-toolbox/internal/tools/looker/lookercommon"
"github.com/googleapis/genai-toolbox/internal/util"
"github.com/looker-open-source/sdk-codegen/go/rtl"
v4 "github.com/looker-open-source/sdk-codegen/go/sdk/v4"
)
// =================================================================================================================
// START MCP SERVER CORE LOGIC
// =================================================================================================================
const kind string = "looker-health-vacuum"
func init() {
if !tools.Register(kind, newConfig) {
panic(fmt.Sprintf("tool kind %q already registered", kind))
}
}
func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.ToolConfig, error) {
actual := Config{Name: name}
if err := decoder.DecodeContext(ctx, &actual); err != nil {
return nil, err
}
return actual, nil
}
type Config struct {
Name string `yaml:"name" validate:"required"`
Kind string `yaml:"kind" validate:"required"`
Source string `yaml:"source" validate:"required"`
Description string `yaml:"description" validate:"required"`
AuthRequired []string `yaml:"authRequired"`
Parameters map[string]any `yaml:"parameters"`
}
var _ tools.ToolConfig = Config{}
func (cfg Config) ToolConfigKind() string {
return kind
}
func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error) {
rawS, ok := srcs[cfg.Source]
if !ok {
return nil, fmt.Errorf("no source named %q configured", cfg.Source)
}
s, ok := rawS.(*lookersrc.Source)
if !ok {
return nil, fmt.Errorf("invalid source for %q tool: source kind must be `looker`", kind)
}
actionParameter := tools.NewStringParameterWithRequired("action", "The vacuum action to run. Can be 'models', or 'explores'.", true)
projectParameter := tools.NewStringParameterWithDefault("project", "", "The Looker project to vacuum (optional).")
modelParameter := tools.NewStringParameterWithDefault("model", "", "The Looker model to vacuum (optional).")
exploreParameter := tools.NewStringParameterWithDefault("explore", "", "The Looker explore to vacuum (optional).")
timeframeParameter := tools.NewIntParameterWithDefault("timeframe", 90, "The timeframe in days to analyze.")
minQueriesParameter := tools.NewIntParameterWithDefault("min_queries", 1, "The minimum number of queries for a model or explore to be considered used.")
parameters := tools.Parameters{
actionParameter,
projectParameter,
modelParameter,
exploreParameter,
timeframeParameter,
minQueriesParameter,
}
mcpManifest := tools.GetMcpManifest(cfg.Name, cfg.Description, cfg.AuthRequired, parameters)
return Tool{
Name: cfg.Name,
Kind: kind,
Parameters: parameters,
AuthRequired: cfg.AuthRequired,
UseClientOAuth: s.UseClientOAuth,
Client: s.Client,
ApiSettings: s.ApiSettings,
manifest: tools.Manifest{
Description: cfg.Description,
Parameters: parameters.Manifest(),
AuthRequired: cfg.AuthRequired,
},
mcpManifest: mcpManifest,
}, nil
}
var _ tools.Tool = Tool{}
type Tool struct {
Name string `yaml:"name"`
Kind string `yaml:"kind"`
UseClientOAuth bool
Client *v4.LookerSDK
ApiSettings *rtl.ApiSettings
AuthRequired []string `yaml:"authRequired"`
Parameters tools.Parameters
manifest tools.Manifest
mcpManifest tools.McpManifest
}
func (t Tool) Invoke(ctx context.Context, params tools.ParamValues, accessToken tools.AccessToken) (any, error) {
sdk, err := lookercommon.GetLookerSDK(t.UseClientOAuth, t.ApiSettings, t.Client, accessToken)
if err != nil {
return nil, fmt.Errorf("error getting sdk: %w", err)
}
paramsMap := params.AsMap()
timeframe, _ := paramsMap["timeframe"].(int)
if timeframe == 0 {
timeframe = 90
}
minQueries, _ := paramsMap["min_queries"].(int)
if minQueries == 0 {
minQueries = 1
}
vacuumTool := &vacuumTool{
SdkClient: sdk,
timeframe: timeframe,
minQueries: minQueries,
}
action, ok := paramsMap["action"].(string)
if !ok {
return nil, fmt.Errorf("action parameter not found")
}
switch action {
case "models":
project, _ := paramsMap["project"].(string)
model, _ := paramsMap["model"].(string)
return vacuumTool.models(ctx, project, model)
case "explores":
model, _ := paramsMap["model"].(string)
explore, _ := paramsMap["explore"].(string)
return vacuumTool.explores(ctx, model, explore)
default:
return nil, fmt.Errorf("unknown action: %s", action)
}
}
func (t Tool) ParseParams(data map[string]any, claims map[string]map[string]any) (tools.ParamValues, error) {
return tools.ParseParams(t.Parameters, data, claims)
}
func (t Tool) Manifest() tools.Manifest {
return t.manifest
}
func (t Tool) McpManifest() tools.McpManifest {
return t.mcpManifest
}
func (t Tool) Authorized(verifiedAuthServices []string) bool {
return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices)
}
func (t Tool) RequiresClientAuthorization() bool {
return t.UseClientOAuth
}
// =================================================================================================================
// END MCP SERVER CORE LOGIC
// =================================================================================================================
// =================================================================================================================
// START LOOKER HEALTH VACUUM CORE LOGIC
// =================================================================================================================
type vacuumTool struct {
SdkClient *v4.LookerSDK
timeframe int
minQueries int
}
func (t *vacuumTool) models(ctx context.Context, project, model string) ([]map[string]interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Vacuuming models...")
usedModels, err := t.getUsedModels(ctx)
if err != nil {
return nil, err
}
lookmlModels, err := t.SdkClient.AllLookmlModels(v4.RequestAllLookmlModels{}, nil)
if err != nil {
return nil, fmt.Errorf("error fetching LookML models: %w", err)
}
var results []map[string]interface{}
for _, m := range lookmlModels {
if (project == "" || (m.ProjectName != nil && *m.ProjectName == project)) &&
(model == "" || (m.Name != nil && *m.Name == model)) {
queryCount := 0
if qc, ok := usedModels[*m.Name]; ok {
queryCount = qc
}
unusedExplores, err := t.getUnusedExplores(ctx, *m.Name)
if err != nil {
return nil, err
}
results = append(results, map[string]interface{}{
"Model": *m.Name,
"Unused Explores": unusedExplores,
"Model Query Count": queryCount,
})
}
}
return results, nil
}
func (t *vacuumTool) explores(ctx context.Context, model, explore string) ([]map[string]interface{}, error) {
logger, err := util.LoggerFromContext(ctx)
if err != nil {
return nil, fmt.Errorf("unable to get logger from ctx: %s", err)
}
logger.InfoContext(ctx, "Vacuuming explores...")
lookmlModels, err := t.SdkClient.AllLookmlModels(v4.RequestAllLookmlModels{}, nil)
if err != nil {
return nil, fmt.Errorf("error fetching LookML models: %w", err)
}
var results []map[string]interface{}
for _, m := range lookmlModels {
if model != "" && (m.Name == nil || *m.Name != model) {
continue
}
if m.Explores == nil {
continue
}
for _, e := range *m.Explores {
if explore != "" && (e.Name == nil || *e.Name != explore) {
continue
}
if e.Name == nil {
continue
}
exploreDetail, err := t.SdkClient.LookmlModelExplore(v4.RequestLookmlModelExplore{
LookmlModelName: *m.Name,
ExploreName: *e.Name,
}, nil)
if err != nil {
logger.ErrorContext(ctx, fmt.Sprintf("Error fetching detail for explore %s.%s: %v", *m.Name, *e.Name, err))
continue
}
usedFields, err := t.getUsedExploreFields(ctx, *m.Name, *e.Name)
if err != nil {
logger.ErrorContext(ctx, fmt.Sprintf("Error fetching used fields for explore %s.%s: %v", *m.Name, *e.Name, err))
continue
}
var allFields []string
if exploreDetail.Fields != nil {
for _, d := range *exploreDetail.Fields.Dimensions {
if !*d.Hidden {
allFields = append(allFields, *d.Name)
}
}
for _, ms := range *exploreDetail.Fields.Measures {
if !*ms.Hidden {
allFields = append(allFields, *ms.Name)
}
}
}
var unusedFields []string
for _, field := range allFields {
if _, ok := usedFields[field]; !ok {
unusedFields = append(unusedFields, field)
}
}
joinStats := make(map[string]int)
if exploreDetail.Joins != nil {
for field, queryCount := range usedFields {
join := strings.Split(field, ".")[0]
joinStats[join] += queryCount
}
for _, join := range *exploreDetail.Joins {
if _, ok := joinStats[*join.Name]; !ok {
joinStats[*join.Name] = 0
}
}
}
var unusedJoins []string
for join, count := range joinStats {
if count == 0 {
unusedJoins = append(unusedJoins, join)
}
}
results = append(results, map[string]interface{}{
"Model": *m.Name,
"Explore": *e.Name,
"Unused Joins": unusedJoins,
"Unused Fields": unusedFields,
})
}
}
return results, nil
}
func (t *vacuumTool) getUsedModels(ctx context.Context) (map[string]int, error) {
limit := "5000"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"history.query_run_count", "query.model"},
Filters: &map[string]any{
"history.created_date": fmt.Sprintf("%d days", t.timeframe),
"query.model": "-system__activity, -i__looker",
"history.query_run_count": fmt.Sprintf(">%d", t.minQueries-1),
"user.dev_branch_name": "NULL",
},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", nil)
if err != nil {
return nil, err
}
var data []map[string]interface{}
_ = json.Unmarshal([]byte(raw), &data)
results := make(map[string]int)
for _, row := range data {
model, _ := row["query.model"].(string)
count, _ := row["history.query_run_count"].(float64)
results[model] = int(count)
}
return results, nil
}
func (t *vacuumTool) getUnusedExplores(ctx context.Context, modelName string) ([]string, error) {
lookmlModel, err := t.SdkClient.LookmlModel(modelName, "", nil)
if err != nil {
return nil, fmt.Errorf("error fetching LookML model %s: %w", modelName, err)
}
var unusedExplores []string
if lookmlModel.Explores != nil {
for _, e := range *lookmlModel.Explores {
limit := "1"
queryCountQueryBody := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"history.query_run_count"},
Filters: &map[string]any{
"query.model": modelName,
"query.view": *e.Name,
"history.created_date": fmt.Sprintf("%d days", t.timeframe),
"history.query_run_count": fmt.Sprintf(">%d", t.minQueries-1),
"user.dev_branch_name": "NULL",
},
Limit: &limit,
}
rawQueryCount, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, queryCountQueryBody, "json", nil)
if err != nil {
// Log the error but continue
continue
}
var data []map[string]interface{}
_ = json.Unmarshal([]byte(rawQueryCount), &data)
if len(data) == 0 {
unusedExplores = append(unusedExplores, *e.Name)
}
}
}
return unusedExplores, nil
}
func (t *vacuumTool) getUsedExploreFields(ctx context.Context, model, explore string) (map[string]int, error) {
limit := "5000"
query := &v4.WriteQuery{
Model: "system__activity",
View: "history",
Fields: &[]string{"query.formatted_fields", "query.filters", "history.query_run_count"},
Filters: &map[string]any{
"history.created_date": fmt.Sprintf("%d days", t.timeframe),
"query.model": strings.ReplaceAll(model, "_", "^_"),
"query.view": strings.ReplaceAll(explore, "_", "^_"),
"query.formatted_fields": "-NULL",
"history.workspace_id": "production",
},
Limit: &limit,
}
raw, err := lookercommon.RunInlineQuery(ctx, t.SdkClient, query, "json", nil)
if err != nil {
return nil, err
}
var data []map[string]interface{}
_ = json.Unmarshal([]byte(raw), &data)
results := make(map[string]int)
fieldRegex := regexp.MustCompile(`(\w+\.\w+)`)
for _, row := range data {
count, _ := row["history.query_run_count"].(float64)
formattedFields, _ := row["query.formatted_fields"].(string)
filters, _ := row["query.filters"].(string)
usedFields := make(map[string]bool)
for _, field := range fieldRegex.FindAllString(formattedFields, -1) {
results[field] += int(count)
usedFields[field] = true
}
for _, field := range fieldRegex.FindAllString(filters, -1) {
if _, ok := usedFields[field]; !ok {
results[field] += int(count)
}
}
}
return results, nil
}
// =================================================================================================================
// END LOOKER HEALTH VACUUM CORE LOGIC
// =================================================================================================================

View File

@@ -0,0 +1,113 @@
// Copyright 2025 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package lookerhealthvacuum_test
import (
"strings"
"testing"
yaml "github.com/goccy/go-yaml"
"github.com/google/go-cmp/cmp"
"github.com/googleapis/genai-toolbox/internal/server"
"github.com/googleapis/genai-toolbox/internal/testutils"
lhv "github.com/googleapis/genai-toolbox/internal/tools/looker/lookerhealthvacuum"
)
func TestParseFromYamlLookerHealthVacuum(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
want server.ToolConfigs
}{
{
desc: "basic example",
in: `
tools:
example_tool:
kind: looker-health-vacuum
source: my-instance
description: some description
`,
want: server.ToolConfigs{
"example_tool": lhv.Config{
Name: "example_tool",
Kind: "looker-health-vacuum",
Source: "my-instance",
Description: "some description",
AuthRequired: []string{},
},
},
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
// Parse contents
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err != nil {
t.Fatalf("unable to unmarshal: %s", err)
}
if diff := cmp.Diff(tc.want, got.Tools); diff != "" {
t.Fatalf("incorrect parse: diff %v", diff)
}
})
}
}
func TestFailParseFromYamlLookerHealthVacuum(t *testing.T) {
ctx, err := testutils.ContextWithNewLogger()
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
tcs := []struct {
desc string
in string
err string
}{
{
desc: "Invalid field",
in: `
tools:
example_tool:
kind: looker-health-vacuum
source: my-instance
invalid_field: true
`,
err: "unable to parse tool \"example_tool\" as kind \"looker-health-vacuum\": [2:1] unknown field \"invalid_field\"",
},
}
for _, tc := range tcs {
t.Run(tc.desc, func(t *testing.T) {
got := struct {
Tools server.ToolConfigs `yaml:"tools"`
}{}
// Parse contents
err := yaml.UnmarshalContext(ctx, testutils.FormatYaml(tc.in), &got)
if err == nil {
t.Fatalf("expect parsing to fail")
}
errStr := err.Error()
if !strings.Contains(errStr, tc.err) {
t.Fatalf("unexpected error string: got %q, want substring %q", errStr, tc.err)
}
})
}
}

View File

@@ -70,6 +70,7 @@ func TestLooker(t *testing.T) {
var args []string
// Write config into a file and pass it to command
toolsFile := map[string]any{
"sources": map[string]any{
"my-instance": sourceConfig,
@@ -130,6 +131,21 @@ func TestLooker(t *testing.T) {
"source": "my-instance",
"description": "Simple tool to test end to end functionality.",
},
"health_pulse": map[string]any{
"kind": "looker-health-pulse",
"source": "my-instance",
"description": "Checks the health of a Looker instance by running a series of checks on the system.",
},
"health_analyze": map[string]any{
"kind": "looker-health-analyze",
"source": "my-instance",
"description": "Provides analysis of a Looker instance's projects, models, or explores.",
},
"health_vacuum": map[string]any{
"kind": "looker-health-vacuum",
"source": "my-instance",
"description": "Vacuums unused content from a Looker instance.",
},
},
}
@@ -617,6 +633,127 @@ func TestLooker(t *testing.T) {
},
},
)
tests.RunToolGetTestByName(t, "health_pulse",
map[string]any{
"health_pulse": map[string]any{
"description": "Checks the health of a Looker instance by running a series of checks on the system.",
"authRequired": []any{},
"parameters": []any{
map[string]any{
"authSources": []any{},
"description": "The health check to run. Can be either: `check_db_connections`, `check_dashboard_performance`,`check_dashboard_errors`,`check_explore_performance`,`check_schedule_failures`, or `check_legacy_features`",
"name": "action",
"required": true,
"type": "string",
},
},
},
},
)
tests.RunToolGetTestByName(t, "health_analyze",
map[string]any{
"health_analyze": map[string]any{
"description": "Provides analysis of a Looker instance's projects, models, or explores.",
"authRequired": []any{},
"parameters": []any{
map[string]any{
"authSources": []any{},
"description": "The analysis to run. Can be 'projects', 'models', or 'explores'.",
"name": "action",
"required": true,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The Looker project to analyze (optional).",
"name": "project",
"required": false,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The Looker model to analyze (optional).",
"name": "model",
"required": false,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The Looker explore to analyze (optional).",
"name": "explore",
"required": false,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The timeframe in days to analyze.",
"name": "timeframe",
"required": false,
"type": "integer",
},
map[string]any{
"authSources": []any{},
"description": "The minimum number of queries for a model or explore to be considered used.",
"name": "min_queries",
"required": false,
"type": "integer",
},
},
},
},
)
tests.RunToolGetTestByName(t, "health_vacuum",
map[string]any{
"health_vacuum": map[string]any{
"description": "Vacuums unused content from a Looker instance.",
"authRequired": []any{},
"parameters": []any{
map[string]any{
"authSources": []any{},
"description": "The vacuum action to run. Can be 'models', or 'explores'.",
"name": "action",
"required": true,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The Looker project to vacuum (optional).",
"name": "project",
"required": false,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The Looker model to vacuum (optional).",
"name": "model",
"required": false,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The Looker explore to vacuum (optional).",
"name": "explore",
"required": false,
"type": "string",
},
map[string]any{
"authSources": []any{},
"description": "The timeframe in days to analyze.",
"name": "timeframe",
"required": false,
"type": "integer",
},
map[string]any{
"authSources": []any{},
"description": "The minimum number of queries for a model or explore to be considered used.",
"name": "min_queries",
"required": false,
"type": "integer",
},
},
},
},
)
wantResult := "{\"label\":\"System Activity\",\"name\":\"system__activity\",\"project_name\":\"system__activity\"}"
tests.RunToolInvokeSimpleTest(t, "get_models", wantResult)
@@ -651,4 +788,22 @@ func TestLooker(t *testing.T) {
wantResult = "null"
tests.RunToolInvokeParametersTest(t, "get_dashboards", []byte(`{"title": "FOO", "desc": "BAR"}`), wantResult)
wantResult = "\"Connection\":\"thelook\""
tests.RunToolInvokeParametersTest(t, "health_pulse", []byte(`{"action": "check_db_connections"}`), wantResult)
wantResult = "[]"
tests.RunToolInvokeParametersTest(t, "health_pulse", []byte(`{"action": "check_schedule_failures"}`), wantResult)
wantResult = "[{\"Feature\":\"Unsupported in Looker (Google Cloud core)\"}]"
tests.RunToolInvokeParametersTest(t, "health_pulse", []byte(`{"action": "check_legacy_features"}`), wantResult)
wantResult = "\"Project\":\"the_look\""
tests.RunToolInvokeParametersTest(t, "health_analyze", []byte(`{"action": "projects"}`), wantResult)
wantResult = "\"Model\":\"the_look\""
tests.RunToolInvokeParametersTest(t, "health_analyze", []byte(`{"action": "explores", "project": "the_look", "model": "the_look", "explore": "inventory_items"}`), wantResult)
wantResult = "\"Model\":\"the_look\""
tests.RunToolInvokeParametersTest(t, "health_vacuum", []byte(`{"action": "models"}`), wantResult)
}