Professional Services

Common solutions and tools developed by Google Cloud's Professional Services team.

Disclaimer

This repository and its contents are not an officially supported Google product.

License

All solutions within this repository are provided under the Apache 2.0 license. Please see the LICENSE file for more detailed terms and conditions.

Examples

The examples folder contains example solutions across a variety of Google Cloud Platform products. Use these solutions as a reference for your own or extend them to fit your particular use case.

• Anthos Service Mesh Multi-Cluster - Solution to federate two private GKE clusters using Anthos Service Mesh.
• Anthos CICD with Gitlab - A step-by-step guide to create an example CI/CD solution using Anthos and Gitlab.
• Audio Content Profiling - A tool that builds a pipeline to scale the process of moderating audio files for inappropriate content using machine learning APIs.
• Bigdata generator - Solution that generates large amounts of data for stress-testing bigdata solutions (e.g BigQuery). For each of the fields you want to generate, you can specify rules for generating their values. The generated data can stored in BigQuery or GCS (Avro, CSV).
• BigQuery Analyze Realtime Reddit Data - Solution to deploy a (reddit) social media data collection architecture on Google Cloud Platform. Analyzes reddit comments in realtime and provides free natural-language processing / sentiment.
• BigQuery Audit Log Dashboard - Solution to help audit BigQuery usage using Data Studio for visualization and a sample SQL script to query the back-end data source consisting of audit logs.
• BigQuery Audit Log Anomaly Detection - Sample of using BigQuery audit logs for automated anomaly detection and outlier analysis. Generates user friendly graphs for quick bq environment analysis.
• BigQuery Automated Email Exports - Serverless solution to automate the sending of BigQuery export results via email on a scheduled interval. The email will contain a link to a signed or unsigned URL, allowing the recipient to view query results as a JSON, CSV, or Avro file.
• BigQuery Automated Schema Management - Command-line utility for automated provisioning and management of BigQuery datasets and tables.
• BigQuery Billing Dashboard - Solution to help displaying billing info using Data Studio for visualization and a sample SQL script to query the back-end billing export table in BigQuery.
• BigQuery Cross Project Slot Monitoring - Solution to help monitoring slot utilization across multiple projects, while breaking down allocation per project.
• BigQuery Data Consolidator - Solution to consolidate data within an organization from multiple projects into one target Dataset/Table where all Source tables are of same schema (like Billing Exports!); specifically useful for data consolidation and further reporting in Cloud FinOps engagements.
• BigQuery DDL Validator - A utility that will read the Legacy DDL and compare it against the previously extracted DDL and produce an output with the name of the objects where the DDL is no longer matching.
• BigQuery Group Sync For Row Level Access - Sample code to synchronize group membership from G Suite/Cloud Identity into BigQuery and join that with your data to control access at row level.
• BigQuery Long Running Optimization Utility - A utility that reads the entire SQL and provides a list of suggestions that would help to optimize the query and avoid the long running issues.
• BigQuery Oracle DDL Migration Utility - Oracle DDL Migration Utility to migrate the tables schema (DDL) from Oracle DB to BigQuery. The utility leverages BigQuery Translation API and offers additional features such as adding partitioning, clustering, metadata columns and prefixes to table names.
• BigQuery Pipeline Utility - Python utility class for defining data pipelines in BigQuery.
• BigQuery Remote Function - It allows user to implement custom services or libraries in languages other than SQL or Javascript which are not part of UDFs. The utility contains sample string format Java code to deploy cloud run gen2 instance and invoke the service from BigQuery using remote function.
• BigQuery Amazon S3 Migration Tool - Bigquery Migration Tool to transfer data from files in Amazon S3 to BigQuery Tables based on configuration provided.
• BigQuery Snowflake TabRle Migration Tool - BigQuery Snowflake Table Migration Tool helps to migrate the table DDL's from Snowflake to BigQuery. The utility leverages BigQuery Translation API and offers additional features such as adding partitioning, clustering, metadata columns and prefixes to table names.
• BigQuery Table Access Pattern Analysis - Sample code to analyse data pipeline optimisation points, by pinpointing suboptimal pipeline scheduling between tables in a data warehouse ELT job.
• BigQuery Tink Toolkit - Python utility class for working with Tink-based cryptography in on-prem or GCP systems in a way