GSP-323: Perform Foundational Data, ML, and AI Tasks in Google Cloud
Overview
Task - 1 : Run a simple Dataflow job
1bq mk lab
2
3gsutil cp gs://cloud-training/gsp323/lab.csv .
4
5cat lab.csv
6
7gsutil cp gs://cloud-training/gsp323/lab.schema .
8
9cat lab.schema
Task - 2 : Run a simple Dataproc job
- This has to be done mannually.
Task - 3 : Run a simple Dataprep job
- This has to be done mannually.
Task - 4 : AI
1gcloud iam service-accounts create my-natlang-sa \
2 --display-name "my natural language service account"
3
4gcloud iam service-accounts keys create ~/key.json \
5 --iam-account my-natlang-sa@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com
6
7export GOOGLE_APPLICATION_CREDENTIALS="/home/$USER/key.json"
8
9gcloud auth activate-service-account my-natlang-sa@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com --key-file=$GOOGLE_APPLICATION_CREDENTIALS
10
11gcloud ml language analyze-entities --content="Old Norse texts portray Odin as one-eyed and long-bearded, frequently wielding a spear named Gungnir and wearing a cloak and a broad hat." > result.json
12
13gcloud auth login
14(Copy the token from the link provided)
15
16
17gsutil cp result.json gs://YOUR_PROJECT-marking/task4-cnl.result
Create an API key and export as API_KEY
variable.
1export API_KEY={Replace with API KEY}
2
3nano request.json
Add this content:
1{
2 "config": {
3 "encoding":"FLAC",
4 "languageCode": "en-US"
5 },
6 "audio": {
7 "uri":"gs://cloud-training/gsp323/task4.flac"
8 }
9}
1curl -s -X POST -H "Content-Type: application/json" --data-binary @request.json \
2"https://speech.googleapis.com/v1/speech:recognize?key=${API_KEY}" > result.json
3
4gsutil cp result.json gs://YOUR_PROJECT-marking/task4-gcs.result
5
6
7gcloud iam service-accounts create quickstart
8
9gcloud iam service-accounts keys create key.json --iam-account quickstart@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com
10
11gcloud auth activate-service-account --key-file key.json
12
13export ACCESS_TOKEN=$(gcloud auth print-access-token)
14
15
16nano request.json
Add this content:
1{
2 "inputUri":"gs://spls/gsp154/video/chicago.mp4",
3 "features": [
4 "TEXT_DETECTION"
5 ]
6}
Now add the following commands on the command line:
1curl -s -H 'Content-Type: application/json' \
2 -H "Authorization: Bearer $ACCESS_TOKEN" \
3 'https://videointelligence.googleapis.com/v1/videos:annotate' \
4 -d @request.json
5
6
7
8curl -s -H 'Content-Type: application/json' -H "Authorization: Bearer $ACCESS_TOKEN" 'https://videointelligence.googleapis.com/v1/operations/OPERATION_FROM_PREVIOUS_REQUEST' > result1.json
9
10
11gsutil cp result1.json gs://YOUR_PROJECT-marking/task4-gvi.result
Congratulations, you're all done with the lab 😄