Compare commits

...

13 Commits

Author SHA1 Message Date
Gabriele De Rosa 0ac692622d
Merge pull request #3 from derogab/dev
Fix cron worker
2021-05-21 18:25:03 +02:00
Gabriele De Rosa 0c345f30e9 Change cron hour 2021-05-21 18:24:41 +02:00
Gabriele De Rosa b195abf771 Fix cron worker 2021-05-21 18:21:32 +02:00
Gabriele De Rosa bfc8ff46b8
Merge pull request #2 from derogab/dev
Minor Improvements
2021-05-21 15:57:35 +02:00
Gabriele De Rosa 07b8aa59f8 Improve intro 2021-05-21 15:55:23 +02:00
Gabriele De Rosa e74e671ff6 Remove progress bars 2021-05-21 15:51:10 +02:00
Gabriele De Rosa 69d9d57832
Merge pull request #1 from derogab/dev
Restyle
2021-05-21 15:19:16 +02:00
Gabriele De Rosa 6ea9453626 Minor improvements 2021-05-21 15:08:13 +02:00
Gabriele De Rosa d6a93dfd91 Update README.md 2021-05-21 14:50:57 +02:00
Gabriele De Rosa e8a11e9d42 Update README.md 2021-05-21 14:49:56 +02:00
Gabriele De Rosa 0a5038306e Total --> First 2021-05-21 14:25:23 +02:00
Gabriele De Rosa ee5df0c4c7 Restyle 2021-05-21 14:15:34 +02:00
Gabriele De Rosa 374d90b8e3 Minor improvements 2021-05-21 03:28:16 +02:00
6 changed files with 200 additions and 98 deletions

View File

@ -23,4 +23,4 @@ Il bot è stato sviluppato da [@derogab](https://github.com/derogab) e il codice
I dati mostrati sono scaricati dagli [Open Data ufficiali](https://github.com/italia/covid19-opendata-vaccini) sui vaccini in Italia.
I grafici sono automaticamente generati mediante il codice della [repository pubblica](https://github.com/MarcoBuster/quanto-manca) di [@MarcoBuster](https://github.com/MarcoBuster).
Fino alla [v1.0.0](https://github.com/derogab/ITAvsCOVIDbot/releases/tag/v1.0.0) i grafici sono automaticamente generati mediante il codice della [repository pubblica](https://github.com/MarcoBuster/quanto-manca) di [@MarcoBuster](https://github.com/MarcoBuster). La [v2.0.0](https://github.com/derogab/ITAvsCOVIDbot/releases/tag/v2.0.0) introduce invece un aggiornamento di questi grafici.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 43 KiB

View File

@ -26,10 +26,13 @@
padding-top: 0px !important;
margin-bottom: 5px;
}
#content {
padding: 15px 35px;
}
</style>
</head>
<body>
<div class="container mt-2 pt-2 px-xl-5">
<div class="container mt-2 pt-2 px-xl-5" id="content">
<h1>Quando torneremo alla normalità?</h1>
<p>
La domanda non ha una risposta precisa.
@ -41,33 +44,11 @@
</p>
<h3>Analisi dei dati della campagna vaccinale</h3>
<p class="mb-2">
La campagna vaccinale in Italia è iniziata da poco e chiaramente non è ancora a regime. La stima qui effettuata
sarà via via più affidabile con l'aumentare dei dati e indica il tempo che mancherebbe al raggiungimento
La stima qui effettuata indica il tempo che mancherebbe al raggiungimento
dell'immunità di gregge qualora si tenesse il ritmo degli ultimi 7 giorni. <br />
</p>
<hr>
<div class="text-center">
In Italia sono state somministrate <br />
<span class="text-big">
<!-- totalDoses -->
dosi di vaccino</span> <br />
che corrispondono a <br />
<span class="text-big">
<!-- totalVaccinations -->
persone vaccinate,</span> <br />
ovvero <br />
<span class="text-big">
<!-- totalVaccinationsPerc -->
della popolazione.
</span> <br />
Ne abbiamo vaccinate <br />
<span class="text-big">
<!-- totalVaccinationsLastWeek -->
nell'ultima settimana,</span> <br />
con un ritmo di <br />
<span class="text-big">
<!-- vaccinesPerDay -->
vaccinati al giorno.</span> <br />
Continuando di questo passo, raggiungeremo l'immunità di gregge il <br />
<span class="text-bigger">
<!-- hitDate -->
@ -81,13 +62,13 @@
giorni.
</span>
</div>
<img alt="Grafico vaccinati ultima settimana" src="plot.png" class="img-fluid">
<hr>
<p class="pt-2 notes">
I dati non comprendono quelli del giorno attuale perché solitamente incompleti.
</p>
<p class="pt-2 notes">
Dal 2 febbraio 2021 il grafico mostra i dati delle persone vaccinate,
ovvero di chi ha ricevuto entrambe le dosi.
Dal 21 maggio 2021 sono mostrati due differenti grafici per differenziare
le prime e le seconde dosi.
</p>
</div>
</body>

View File

@ -0,0 +1,88 @@
<!DOCTYPE html>
<!-- Source: https://github.com/MarcoBuster/quanto-manca/blob/master/template.html -->
<html lang="it">
<head>
<title>@itavscovidbot</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.0-beta1/dist/css/bootstrap.min.css" rel="stylesheet"
integrity="sha384-giJF6kkoqNQ00vy+HMDP7azOuL0xtbfIcaT9wjKHr8RbDVddVHyTfAAsrekwKmP1" crossorigin="anonymous">
<style>
.text-big {
font-size: 130%;
font-weight: bold;
position: relative;
top: -5px;
margin-bottom: -4px;
}
.text-bigger {
font-size: 200%;
font-weight: bold;
position: relative;
top: -6px;
margin-bottom: -4px;
}
p.notes{
padding-top: 0px !important;
margin-bottom: 5px;
}
.box {
text-align: center;
padding: 10px 0px;
margin: 5px -7px;
border: 1px solid black;
border-radius: 5px;
}
#content {
padding: 35px;
}
</style>
</head>
<body>
<div class="container mt-2 pt-2 px-xl-5" id="content">
<div class="row">
<div class="col-md-6">
<div class="box">
<h1>
<!-- totalVaccinations -->
</h1>
Somministrazioni
<!-- typeVaccinations -->
</div>
</div>
<div class="col-md-6">
<div class="box">
<h1>
<!-- totalVaccinationsPerc -->
</h1>
/ 100%
</div>
</div>
</div>
<div class="row">
<div class="col-md-6">
<div class="box">
<h1>
<!-- totalVaccinationsLastWeek -->
</h1>
Questa settimana
</div>
</div>
<div class="col-md-6">
<div class="box">
<h1>
<!-- vaccinesPerDay -->
</h1>
Media settimanale
</div>
</div>
</div>
<hr>
<img alt="Grafico vaccinati ultima settimana" src="plot.png" class="img-fluid">
</div>
</body>
</html>

149
bot.py
View File

@ -5,6 +5,7 @@ import secrets
import schedule
import time
from dotenv import load_dotenv
from telegram import InputMediaPhoto
from telegram.ext import Updater
from telegram.ext import CommandHandler
from pymongo import MongoClient
@ -52,7 +53,7 @@ mongodb_pass = os.environ.get('MONGODB_PASS')
if mongodb_user is None or mongodb_pass is None:
print('No mongodb auth.')
exit()
mongodb_uri = 'mongodb://' + mongodb_user + ':' + mongodb_pass + '@db:27017',
mongodb_uri = 'mongodb://' + mongodb_user + ':' + mongodb_pass + '@db:27017'
# Init mongodb database
client = MongoClient(mongodb_uri)
@ -61,59 +62,27 @@ db = client['bot']
# Function to generate images from templates
def generate(df, target, template):
# Function to get data
def download():
# Download from open data
r = requests.get(DATA_URL)
# Create dataframe from data
df = pd.read_csv(
io.StringIO(r.text),
index_col="data_somministrazione",
)
# Set date as index
df.index = pd.to_datetime(
df.index,
format="%Y-%m-%d",
)
# Sort value based on date
df.sort_values('data_somministrazione')
# Delete sum data if already exists
df = df[df["area"] != "ITA"]
# Set target counters to numeric
df["totale"] = pd.to_numeric(df["totale"])
df["seconda_dose"] = pd.to_numeric(df["seconda_dose"])
# Group by day and sum counters
df_doses = df.groupby(['data_somministrazione'])['totale'].sum().reset_index()
df_vaccines = df.groupby(['data_somministrazione'])['seconda_dose'].sum().reset_index()
# Re-set date as ID in new dataframe
df_doses = df_doses.set_index('data_somministrazione')
df_vaccines = df_vaccines.set_index('data_somministrazione')
# If there are current day data...
if dt.now() - df_doses.index[-1] < td(days=1):
df_doses = df_doses[:-1] # Ignore the current day because it's often incomplete
if dt.now() - df_vaccines.index[-1] < td(days=1):
df_vaccines = df_vaccines[:-1] # Ignore the current day because it's often incomplete
totalDoses = sum(df_doses["totale"])
totalVaccines = sum(df_vaccines["seconda_dose"])
lastWeekData = df_vaccines.loc[df_vaccines.index > df_vaccines.index[-1] - td(days=7)]
vaccinesPerDayAverage = sum(lastWeekData["seconda_dose"]) / 7
# Get data from df
totalVaccines = sum(df[target])
lastWeekData = df.loc[df.index > df.index[-1] - td(days=7)]
vaccinesPerDayAverage = sum(lastWeekData[target]) / 7
remainingDays = (HIT - totalVaccines) / vaccinesPerDayAverage
hitDate = df_vaccines.index[-1] + td(days=remainingDays)
hitDate = df.index[-1] + td(days=remainingDays)
first_or_second = 'Seconda Dose' if target == 'seconda_dose' else 'Prima Dose' if target == 'prima_dose' else 'Totale'
# Generate plot
plt.ylabel("Vaccinati al giorno")
plt.ylabel(first_or_second)
plt.xlabel("Ultima settimana")
plt.grid(True)
plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
plt.gca().xaxis.set_major_locator(mdates.AutoDateLocator())
plt.gcf().autofmt_xdate()
plt.bar(lastWeekData.index, height=lastWeekData["seconda_dose"])
plt.bar(lastWeekData.index, height=lastWeekData[target])
# Trendline
z = np.polyfit(range(0, 7), lastWeekData["seconda_dose"], 2)
z = np.polyfit(range(0, 7), lastWeekData[target], 2)
p = np.poly1d(z)
plt.plot(lastWeekData.index, p(range(0, 7)), "r--")
# Secret 4 filenames
@ -127,13 +96,13 @@ def download():
# Generate tmp webpage/html filename
webpage_filename = 'tmp_' + sf + '.html'
# Generate template
with open('template.html', 'r+') as f:
with open(template, 'r+') as f:
with open('out/' + webpage_filename, 'w+') as wf:
for line in f.read().splitlines():
if "<!-- totalDoses -->" in line:
line = f"{totalDoses}"
elif "<!-- totalVaccinations -->" in line:
if "<!-- totalVaccinations -->" in line:
line = f"{totalVaccines}"
elif "<!-- typeVaccinations -->" in line:
line = f"{first_or_second}"
elif "<!-- totalVaccinationsPerc -->" in line:
line = f"{str(round(totalVaccines / ITALIAN_POPULATION * 100, 2)).replace('.', ',')}%"
elif "<!-- totalVaccinationsLastWeek -->" in line:
@ -162,6 +131,51 @@ def download():
}
# Function to get data
def download():
# Download from open data
r = requests.get(DATA_URL)
# Create dataframe from data
df = pd.read_csv(
io.StringIO(r.text),
index_col="data_somministrazione",
)
# Set date as index
df.index = pd.to_datetime(
df.index,
format="%Y-%m-%d",
)
# Sort value based on date
df.sort_values('data_somministrazione')
# Delete sum data if already exists
df = df[df["area"] != "ITA"]
# Set target counters to numeric
df["prima_dose"] = pd.to_numeric(df["prima_dose"])
df["seconda_dose"] = pd.to_numeric(df["seconda_dose"])
# Group by day and sum counters
df_first = df.groupby(['data_somministrazione'])['prima_dose'].sum().reset_index()
df_second = df.groupby(['data_somministrazione'])['seconda_dose'].sum().reset_index()
# Re-set date as ID in new dataframe
df_first = df_first.set_index('data_somministrazione')
df_second = df_second.set_index('data_somministrazione')
# If there are current day data...
if dt.now() - df_first.index[-1] < td(days=1):
df_first = df_first[:-1] # Ignore the current day because it's often incomplete
if dt.now() - df_second.index[-1] < td(days=1):
df_second = df_second[:-1] # Ignore the current day because it's often incomplete
# Generata images
intro = generate(df_second, 'seconda_dose', 'assets/templates/intro.html')
plot1 = generate(df_first, 'prima_dose', 'assets/templates/plot.html')
plot2 = generate(df_second, 'seconda_dose', 'assets/templates/plot.html')
return intro, plot1, plot2
# Help command
def help(update, context):
@ -179,7 +193,6 @@ def help(update, context):
# Info command
def info(update, context):
@ -193,6 +206,7 @@ def info(update, context):
context.bot.send_message(chat_id=update.effective_chat.id, text=info_msg, parse_mode='Markdown', disable_web_page_preview=True)
# Start command
def start(update, context):
global db
@ -225,6 +239,7 @@ def start(update, context):
info(update, context)
# Stop command
def stop(update, context):
global db
@ -247,6 +262,8 @@ def stop(update, context):
# Send welcome message
context.bot.send_message(chat_id=update.effective_chat.id, text=stop_msg, parse_mode='Markdown')
# News command
def news(update, context):
global db
@ -287,16 +304,27 @@ def news(update, context):
# Send welcome message
context.bot.send_message(chat_id=update.effective_chat.id, text=set_msg, parse_mode='Markdown')
# Get data
def get(update, context):
# Download data
data = download()
intro, plot1, plot2 = download()
# Send photo
context.bot.send_photo(chat_id=update.effective_chat.id, photo=open(data['results'], 'rb'), caption="")
# Send photos
p1 = open(intro['results'], 'rb')
p2 = open(plot1['results'], 'rb')
p3 = open(plot2['results'], 'rb')
p1 = InputMediaPhoto(media=p1)
p2 = InputMediaPhoto(media=p2)
p3 = InputMediaPhoto(media=p3)
context.bot.send_media_group(chat_id=update.effective_chat.id, media=[p1, p2, p3])
# Remove tmp files
for data in [intro, plot1, plot2]:
os.remove(data['plot'])
os.remove(data['webpage'])
os.remove(data['results'])
@ -311,7 +339,7 @@ def job():
print('Job running...')
# Download data
data = download()
intro, plot1, plot2 = download()
# Get active user with daily news
users = db.users.find({
@ -326,8 +354,16 @@ def job():
for user in users:
try:
# Send photo
updater.bot.send_photo(chat_id=user['_id'], photo=open(data['results'], 'rb'), caption="")
# Send photos
p1 = open(intro['results'], 'rb')
p2 = open(plot1['results'], 'rb')
p3 = open(plot2['results'], 'rb')
p1 = InputMediaPhoto(media=p1)
p2 = InputMediaPhoto(media=p2)
p3 = InputMediaPhoto(media=p3)
updater.bot.send_media_group(chat_id=user['_id'], media=[p1, p2, p3])
# Count news successfully sent
news_sent_counter = news_sent_counter + 1
# Log
@ -342,6 +378,7 @@ def job():
print("[JOB] Results: {sent} messages sent, {fail} messages failed.".format(sent=news_sent_counter, fail=news_fail_counter))
# Remove tmp files
for data in [intro, plot1, plot2]:
os.remove(data['plot'])
os.remove(data['webpage'])
os.remove(data['results'])
@ -373,7 +410,7 @@ start_handler = CommandHandler('news', news)
dispatcher.add_handler(start_handler)
# Setup cron
schedule.every().day.at("18:00").do(job)
schedule.every().day.at("18:30").do(job)
# Start bot
updater.start_polling()

View File

@ -1,14 +1,15 @@
version: "3.5"
version: '3.7'
networks:
itavscovidbot.network:
name: itavscovidbot.network
driver: bridge
services:
db:
image: mongo
image: mongo:latest
container_name: itavscovidbot.db
restart: unless-stopped
env_file:
@ -21,11 +22,9 @@ services:
db.web:
depends_on:
- db
image: mongo-express
image: mongo-express:latest
container_name: itavscovidbot.db.web
restart: unless-stopped
links:
- db
env_file:
- ./db/web/.env
ports:
@ -36,13 +35,10 @@ services:
app:
depends_on:
- db
- db.web
build:
context: .
container_name: itavscovidbot.app
restart: unless-stopped
links:
- db
env_file:
- ./.env
networks: