Python, Lambda, Layers & Rain
Even though I’ve spent years architecting infrastructure, there are times I do miss software development. Not to date myself too much, but yes it was mostly in FORTRAN on a DEC/VAX, IBM/3270 Assembler code with a smattering of VB.Net and C++ later down the road. In case your wondering, yes, I have also gone to IT/Operations desk with a rubberbanded set of punch cards that were invetiably out of order and had to wait a day to resubmit. Yuck!
I thought what better way to resharpen some lost development skills than by learning Python. I’m currently working my way through Angela Yu’s 100 Days of Code – The Complete Python Pro Bootcamp. I’m not disappointed at all. Python has really great features and I’m learning more each day. There is a great section in the course where we are learning all about APIs that you can call from Python to do most anything you can think of.
In this section, the coding challenge was to pull the current weather from the Current Weather Data API at OpenWeather.org analyze the forecast and using Twilio SMS services send yourself an umbrella reminder text if it’s going to rain. Since it has been raining here for what seems like the last month, I thought this could come in handy.
The code is fairly simple but running it from my MacBook Air each morning is probably not the most efficient way to execute it. Immediately AWS Lambda came to mind. After a small amount of tweaking, I got the code up and running in no time. Even though I was getting the texts and the code was executing successfully, Lambda was returning this warning:
Function Logs
START RequestId: 8a22bd2c-133a-4ed4-87f0-77d3138111bb Version: $LATEST
/var/runtime/botocore/vendored/requests/api.py:72: DeprecationWarning: You are using the get() function from 'botocore.vendored.requests'. This dependency was removed from Botocore and will be removed from Lambda after 2021/03/31. https://aws.amazon.com/blogs/developer/removing-the-vendored-version-of-requests-from-botocore/. Install the requests package, 'import requests' directly, and use the requests.get() function instead.
DeprecationWarning
The code as I originally wrote it was using the Python requests module. I’m fairly certain there is an easy conversion from the requests module to something like urllib, but that made me wonder what if there was a package(s) that my code really needed that wasn’t available by default? Something like pandas perhaps. Step in Lambda Layers!
Stefan French has a great getting started article: “Python packages in AWS Lambda made easy” Easy is what I always need.
The basic steps to create your Lambda layer:
- Start a Cloud9 Linux instance. In my case, I just need a very small t2.micro instance using Amazon Linux.
- From the terminal window on your Cloud9 instance run these commands:
mkdir layers
cd layers
virtualenv v-env
source ./v-env/bin/activate
pip install <your package(s)>
deactivate
mkdir python
cd python
cp -r ../v-env/lib64/python3.7/site-packages * .
cd ..
zip -r lambda_layer.zip python
aws lambda publish-layer-version --layer-name <your layer name> --zip-file fileb://lambda_layer.zip --compatible-runtimes python3.7
- Now we simply add our layer into the lambda function
I changed the import statement back to the “requests” module. Tested the code again and the warnings are gone…
Response
"Weather Checked Successfully!"
The only thing to do now is to create a CloudWatch event to run my Lambda function once a day at 7:00AM (EST) and I should be good to go…
If anyone is taking the course, I don’t want to spoil it for you, so stop scrolling.. otherwise enjoy. You will need to set your own environment variables for your latitude and longitude. You can get them from latlong.net address lookup along with your own API keys for OpenWeatherMap and Twilio. The next turn of the crank is to move those keys into AWS Secrets Manager.
def lambda_handler(event, context):
MY_API = os.environ.get("MY_API")
MY_LAT = os.environ.get("MY_LAT")
MY_LNG = os.environ.get("MY_LNG")
TO_NUMBER = os.environ.get("TO_NUMBER")
FROM_NUMBER = os.environ.get("FROM_NUMBER")
TWILIO_ACCOUNT_SID = os.environ.get("TWILIO_SID")
TWILIO_AUTH_TOKEN = os.environ.get("TWILIO_TOKEN")
TWILIO_SMS_URL = "https://api.twilio.com/2010-04-01/Accounts/{}/Messages.json"
OWM_ENDPOINT = "https://api.openweathermap.org/data/2.5/onecall"
body="Hey Mike...It's going to rain today, bring an ☔"
parameters = {
"lat": MY_LAT,
"lon": MY_LNG,
"exclude": "current,minutely,daily",
"appid": MY_API,
}
# Retrieve the forecast
response = requests.get(url=OWM_ENDPOINT, params=parameters)
response.raise_for_status()
weather_data = response.json()
weather_slice = weather_data["hourly"][:12]
will_rain = False
for hour_data in weather_slice:
condition_code = hour_data["weather"][0]["id"]
if int(condition_code) < 700:
will_rain = True
if will_rain:
# insert Twilio Account SID into the REST API URL
populated_url = TWILIO_SMS_URL.format(TWILIO_ACCOUNT_SID)
post_params = {"To": TO_NUMBER, "From": FROM_NUMBER, "Body": body}
# encode the parameters for Python's urllib
data = parse.urlencode(post_params).encode()
req = request.Request(populated_url)
# add authentication header to request based on Account SID + Auth Token
authentication = "{}:{}".format(TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN)
base64string = base64.b64encode(authentication.encode('utf-8'))
req.add_header("Authorization", "Basic %s" % base64string.decode('ascii'))
try:
# perform HTTP POST request
with request.urlopen(req, data) as f:
print("Twilio returned {}".format(str(f.read().decode('utf-8'))))
except Exception as e:
# something went wrong!
return e
return "Weather Checked Successfully!"
Never stop learning and Cloud On!
The cloud is an architect’s dream. Prior to the cloud if I screwed something up there was tangible evidence that had to be destroyed. Now it’s just a blip in the bill. – Mike Spence