Description
Assignment 4
CSCI 5410 (Serverless Data Processing)
Tasks:
This assignment has 2 parts. Part A is related to coding, development, and testing. Part B is
an experiment related to GCP BigQueryML
Part A. Use AWS Lambda-SQS-SNS:
take screenshots at every step and submit as part of the PDF:
a. Alice and Emily work at HfxDine, which is a cloud kitchen that provides delivery service. They
receive orders online. Alice delivers food parcels, and Emily prepares it.
b. The customers’ orders are added to a Queue (standard SQS) –
• Assume a program is sending random food order messages to HfxDine
• This message simulates how a customer places an order to an online store. (You can
ignore, price/tax etc)
• The program should randomly pick one combo meal from a list, create a message body,
and send to HfxDine
c. Emily periodically (every 500 ms or any delay of your choice) checks, if there is any order in the
Queue.
d. If message is available, it is assumed Emily has packed the combo meal, and then a notification
service (SNS) is triggered which sends the details to Alice’s email (Your email in this case)
Part A – Submission requirement:
Submit screenshots of every steps. Please do not exclude any steps. Include all screenshots as part of a
PDF file. In addition, provide the program/scripts as part of the PDF file. Submit code as part of PDF and also
upload the code in gitlab.
Part B. Use GCP BigQueryML:
take screenshots at every step and submit as part of the PDF:
a. Read and understand GCP BigQueryML, which you will need to create KMeans cluster of the given
dataset.
https://cloud.google.com/bigquery-ml/docs/reference/standard-sql/bigqueryml-syntax-createkmeans
b. Use the given dataset “SDey_FTP_input.csv” as your training dataset and test dataset to form and
test the model. The dataset contains two columns, vector1 and vector2. Use Euclidean distance as
the distance type to measure distance between the given vectors
Program Standard
Queue
Lambda Poll
SQS
Notification
Service
Fixed Food list containing combo meals
E.g. {(pasta+pop), (pizza+pop) etc.} Random selection
Simulated Customer Emily_lambda Alice
Alice’s Email = Your DAL Email
c. Create a random sampling with 75% of the record that is available in the given file as training set
and remaining 25% as the test set.
d. You can create the two datasets (training and test) before performing the BigQueryML operation.
e. If you need clarification about this specific problem, do not hesitate to ask me (send me email or
message on Teams)
Part B- Submission requirement:
Submit screenshots of every steps. Please do not exclude any steps. Include all screenshots as part of a
PDF file. In addition, provide any program/query as part of the PDF file. Submit any code you write for this
problem as part of PDF and upload the code in gitlab.
