GSP-323: Perform Foundational Data, ML, and AI Tasks in Google Cloud

GSP-323: Perform Foundational Data, ML, and AI Tasks in Google Cloud

Overview

Task - 1 : Run a simple Dataflow job

1bq mk lab
2
3gsutil cp gs://cloud-training/gsp323/lab.csv .
4
5cat lab.csv
6
7gsutil cp gs://cloud-training/gsp323/lab.schema .
8
9cat lab.schema

Task - 2 : Run a simple Dataproc job

  • This has to be done mannually.

Task - 3 : Run a simple Dataprep job

  • This has to be done mannually.

Task - 4 : AI

 1gcloud iam service-accounts create my-natlang-sa \
 2  --display-name "my natural language service account"
 3
 4gcloud iam service-accounts keys create ~/key.json \
 5  --iam-account my-natlang-sa@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com
 6
 7export GOOGLE_APPLICATION_CREDENTIALS="/home/$USER/key.json"
 8
 9gcloud auth activate-service-account my-natlang-sa@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com --key-file=$GOOGLE_APPLICATION_CREDENTIALS
10
11gcloud ml language analyze-entities --content="Old Norse texts portray Odin as one-eyed and long-bearded, frequently wielding a spear named Gungnir and wearing a cloak and a broad hat." > result.json
12
13gcloud auth login
14(Copy the token from the link provided)
15
16
17gsutil cp result.json gs://YOUR_PROJECT-marking/task4-cnl.result

Create an API key and export as API_KEY variable.

1export API_KEY={Replace with API KEY}
2
3nano request.json

Add this content:

1{
2  "config": {
3      "encoding":"FLAC",
4      "languageCode": "en-US"
5  },
6  "audio": {
7      "uri":"gs://cloud-training/gsp323/task4.flac"
8  }
9}
 1curl -s -X POST -H "Content-Type: application/json" --data-binary @request.json \
 2"https://speech.googleapis.com/v1/speech:recognize?key=${API_KEY}" > result.json
 3
 4gsutil cp result.json gs://YOUR_PROJECT-marking/task4-gcs.result
 5
 6
 7gcloud iam service-accounts create quickstart
 8
 9gcloud iam service-accounts keys create key.json --iam-account quickstart@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com
10
11gcloud auth activate-service-account --key-file key.json
12
13export ACCESS_TOKEN=$(gcloud auth print-access-token)
14
15
16nano request.json

Add this content:

1{
2   "inputUri":"gs://spls/gsp154/video/chicago.mp4",
3   "features": [
4       "TEXT_DETECTION"
5   ]
6}

Now add the following commands on the command line:

 1curl -s -H 'Content-Type: application/json' \
 2    -H "Authorization: Bearer $ACCESS_TOKEN" \
 3    'https://videointelligence.googleapis.com/v1/videos:annotate' \
 4    -d @request.json
 5
 6
 7
 8curl -s -H 'Content-Type: application/json' -H "Authorization: Bearer $ACCESS_TOKEN" 'https://videointelligence.googleapis.com/v1/operations/OPERATION_FROM_PREVIOUS_REQUEST' > result1.json
 9
10
11gsutil cp result1.json gs://YOUR_PROJECT-marking/task4-gvi.result

Congratulations, you're all done with the lab 😄