Merge pull request #3 from derogab/dev

Fix cron worker
Change cron hour
2021-05-21 18:25:03 +02:00 · 2021-05-21 18:24:41 +02:00 · 2021-05-21 18:21:32 +02:00 · 2021-05-21 15:57:35 +02:00 · 2021-05-21 15:55:23 +02:00 · 2021-05-21 15:51:10 +02:00
6 changed files with 200 additions and 98 deletions
--- a/README.md
+++ b/README.md
@ -23,4 +23,4 @@ Il bot è stato sviluppato da [@derogab](https://github.com/derogab) e il codice

 I dati mostrati sono scaricati dagli [Open Data ufficiali](https://github.com/italia/covid19-opendata-vaccini) sui vaccini in Italia. 

-I grafici sono automaticamente generati mediante il codice della [repository pubblica](https://github.com/MarcoBuster/quanto-manca) di [@MarcoBuster](https://github.com/MarcoBuster).
+Fino alla [v1.0.0](https://github.com/derogab/ITAvsCOVIDbot/releases/tag/v1.0.0) i grafici sono automaticamente generati mediante il codice della [repository pubblica](https://github.com/MarcoBuster/quanto-manca) di [@MarcoBuster](https://github.com/MarcoBuster). La [v2.0.0](https://github.com/derogab/ITAvsCOVIDbot/releases/tag/v2.0.0) introduce invece un aggiornamento di questi grafici.
--- a/assets/images/logo.png
+++ b/assets/images/logo.png
--- a/assets/templates/intro.html
+++ b/assets/templates/intro.html
@ -26,10 +26,13 @@
 			padding-top: 0px !important;
 			margin-bottom: 5px;
 		}
+		#content {
+			padding: 15px 35px;
+		}
 	</style>
 </head>
 <body>
-	<div class="container mt-2 pt-2 px-xl-5">
+	<div class="container mt-2 pt-2 px-xl-5" id="content">
 		<h1>Quando torneremo alla normalità?</h1>
 		<p>
 			La domanda non ha una risposta precisa.
@ -41,33 +44,11 @@
 		</p>
 		<h3>Analisi dei dati della campagna vaccinale</h3>
 		<p class="mb-2">
-			La campagna vaccinale in Italia è iniziata da poco e chiaramente non è ancora a regime. La stima qui effettuata
-			sarà via via più affidabile con l'aumentare dei dati e indica il tempo che mancherebbe al raggiungimento 
+			La stima qui effettuata indica il tempo che mancherebbe al raggiungimento 
 			dell'immunità di gregge qualora si tenesse il ritmo degli ultimi 7 giorni. <br />
 		</p>
 		<hr>
 		<div class="text-center">
-			In Italia sono state somministrate <br />
-			<span class="text-big">
-				<!-- totalDoses -->
-				dosi di vaccino</span> <br />
-			che corrispondono a <br />
-			<span class="text-big">
-				<!-- totalVaccinations -->
-				persone vaccinate,</span> <br />
-			ovvero <br />
-			<span class="text-big">
-				<!-- totalVaccinationsPerc -->
-				della popolazione.
-			</span> <br />
-			Ne abbiamo vaccinate <br />
-			<span class="text-big">
-				<!-- totalVaccinationsLastWeek -->
-				nell'ultima settimana,</span> <br />
-			con un ritmo di <br />
-			<span class="text-big">
-				<!-- vaccinesPerDay -->
-				vaccinati al giorno.</span> <br />
 			Continuando di questo passo, raggiungeremo l'immunità di gregge il <br />
 			<span class="text-bigger">
 				<!-- hitDate -->
@ -81,13 +62,13 @@
 				giorni.
 			</span>
 		</div>
-		<img alt="Grafico vaccinati ultima settimana" src="plot.png" class="img-fluid">
+		<hr>
 		<p class="pt-2 notes">
 			I dati non comprendono quelli del giorno attuale perché solitamente incompleti.
 		</p>
 		<p class="pt-2 notes">
-			Dal 2 febbraio 2021 il grafico mostra i dati delle persone vaccinate, 
-			ovvero di chi ha ricevuto entrambe le dosi.
+			Dal 21 maggio 2021 sono mostrati due differenti grafici per differenziare 
+			le prime e le seconde dosi. 
 		</p>
 	</div>
 </body>
--- a/assets/templates/plot.html
+++ b/assets/templates/plot.html
@ -0,0 +1,88 @@
+<!DOCTYPE html>
+<!-- Source: https://github.com/MarcoBuster/quanto-manca/blob/master/template.html -->
+<html lang="it">
+<head>
+	<title>@itavscovidbot</title>
+	<meta charset="utf-8">
+	<meta name="viewport" content="width=device-width, initial-scale=1">
+	<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.0-beta1/dist/css/bootstrap.min.css" rel="stylesheet"
+	      integrity="sha384-giJF6kkoqNQ00vy+HMDP7azOuL0xtbfIcaT9wjKHr8RbDVddVHyTfAAsrekwKmP1" crossorigin="anonymous">
+	<style>
+		.text-big {
+			font-size: 130%;
+			font-weight: bold;
+			position: relative;
+			top: -5px;
+			margin-bottom: -4px;
+		}
+        .text-bigger {
+            font-size: 200%;
+            font-weight: bold;
+            position: relative;
+            top: -6px;
+            margin-bottom: -4px;
+        }
+		p.notes{
+			padding-top: 0px !important;
+			margin-bottom: 5px;
+		}
+		.box {
+			text-align: center;
+			padding: 10px 0px;
+			margin: 5px -7px;
+			border: 1px solid black;
+			border-radius: 5px;
+		}
+		#content {
+			padding: 35px;
+		}
+	</style>
+</head>
+<body>
+	<div class="container mt-2 pt-2 px-xl-5" id="content">
+		
+		<div class="row">
+			<div class="col-md-6">
+				<div class="box">
+					<h1>
+						<!-- totalVaccinations -->
+					</h1>
+					Somministrazioni
+					<!-- typeVaccinations -->
+				</div>
+			</div>
+			<div class="col-md-6">
+				<div class="box">
+					<h1>
+						<!-- totalVaccinationsPerc -->
+					</h1>
+					/ 100%
+				</div>
+			</div>
+		</div>
+		<div class="row">
+			<div class="col-md-6">
+				<div class="box">
+					<h1>
+						<!-- totalVaccinationsLastWeek -->
+					</h1>
+					Questa settimana
+				</div>
+			</div>
+			<div class="col-md-6">
+				<div class="box">
+					<h1>
+						<!-- vaccinesPerDay -->
+					</h1>
+					Media settimanale
+				</div>
+			</div>
+		</div>
+
+		<hr>
+
+		<img alt="Grafico vaccinati ultima settimana" src="plot.png" class="img-fluid">
+		
+	</div>
+</body>
+</html>
--- a/bot.py
+++ b/bot.py
@ -5,6 +5,7 @@ import secrets
 import schedule
 import time
 from dotenv import load_dotenv
+from telegram import InputMediaPhoto
 from telegram.ext import Updater
 from telegram.ext import CommandHandler
 from pymongo import MongoClient
@ -52,7 +53,7 @@ mongodb_pass = os.environ.get('MONGODB_PASS')
 if mongodb_user is None or mongodb_pass is None:
    print('No mongodb auth.')
    exit()
-mongodb_uri  = 'mongodb://' + mongodb_user + ':' + mongodb_pass + '@db:27017',
+mongodb_uri  = 'mongodb://' + mongodb_user + ':' + mongodb_pass + '@db:27017'
 # Init mongodb database
 client = MongoClient(mongodb_uri)

@ -61,59 +62,27 @@ db = client['bot']



+# Function to generate images from templates
+def generate(df, target, template):

-# Function to get data
-def download():
-
-    # Download from open data
-    r = requests.get(DATA_URL)
-    # Create dataframe from data
-    df = pd.read_csv(
-        io.StringIO(r.text),
-        index_col="data_somministrazione",
-    )
-    # Set date as index 
-    df.index = pd.to_datetime(
-        df.index,
-        format="%Y-%m-%d",
-    )
-    # Sort value based on date
-    df.sort_values('data_somministrazione')
-    # Delete sum data if already exists
-    df = df[df["area"] != "ITA"]
-    # Set target counters to numeric
-    df["totale"] = pd.to_numeric(df["totale"])
-    df["seconda_dose"] = pd.to_numeric(df["seconda_dose"])
-    # Group by day and sum counters
-    df_doses = df.groupby(['data_somministrazione'])['totale'].sum().reset_index()
-    df_vaccines = df.groupby(['data_somministrazione'])['seconda_dose'].sum().reset_index()
-    # Re-set date as ID in new dataframe
-    df_doses = df_doses.set_index('data_somministrazione')
-    df_vaccines = df_vaccines.set_index('data_somministrazione')
-    
-    # If there are current day data...
-    if dt.now() - df_doses.index[-1] < td(days=1):
-        df_doses = df_doses[:-1]  # Ignore the current day because it's often incomplete
-    if dt.now() - df_vaccines.index[-1] < td(days=1):
-        df_vaccines = df_vaccines[:-1]  # Ignore the current day because it's often incomplete
-
-    totalDoses = sum(df_doses["totale"])
-    totalVaccines = sum(df_vaccines["seconda_dose"])
-    lastWeekData = df_vaccines.loc[df_vaccines.index > df_vaccines.index[-1] - td(days=7)]
-    vaccinesPerDayAverage = sum(lastWeekData["seconda_dose"]) / 7
+    # Get data from df
+    totalVaccines = sum(df[target])
+    lastWeekData = df.loc[df.index > df.index[-1] - td(days=7)]
+    vaccinesPerDayAverage = sum(lastWeekData[target]) / 7
    remainingDays = (HIT - totalVaccines) / vaccinesPerDayAverage
-    hitDate = df_vaccines.index[-1] + td(days=remainingDays)
+    hitDate = df.index[-1] + td(days=remainingDays)
+    first_or_second = 'Seconda Dose' if target == 'seconda_dose' else 'Prima Dose' if target == 'prima_dose' else 'Totale'

    # Generate plot
-    plt.ylabel("Vaccinati al giorno")
+    plt.ylabel(first_or_second)
    plt.xlabel("Ultima settimana")
    plt.grid(True)
    plt.gca().xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
    plt.gca().xaxis.set_major_locator(mdates.AutoDateLocator())
    plt.gcf().autofmt_xdate()
-    plt.bar(lastWeekData.index, height=lastWeekData["seconda_dose"])
+    plt.bar(lastWeekData.index, height=lastWeekData[target])
    # Trendline
-    z = np.polyfit(range(0, 7), lastWeekData["seconda_dose"], 2)
+    z = np.polyfit(range(0, 7), lastWeekData[target], 2)
    p = np.poly1d(z)
    plt.plot(lastWeekData.index, p(range(0, 7)), "r--")
    # Secret 4 filenames
@ -127,13 +96,13 @@ def download():
    # Generate tmp webpage/html filename
    webpage_filename = 'tmp_' + sf + '.html'
    # Generate template
-    with open('template.html', 'r+') as f:
+    with open(template, 'r+') as f:
        with open('out/' + webpage_filename, 'w+') as wf:
            for line in f.read().splitlines():
-                if "<!-- totalDoses -->" in line:
-                    line = f"{totalDoses}"
-                elif "<!-- totalVaccinations -->" in line:
+                if "<!-- totalVaccinations -->" in line:
                    line = f"{totalVaccines}"
+                elif "<!-- typeVaccinations -->" in line:
+                    line = f"{first_or_second}"
                elif "<!-- totalVaccinationsPerc -->" in line:
                    line = f"{str(round(totalVaccines / ITALIAN_POPULATION * 100, 2)).replace('.', ',')}%"
                elif "<!-- totalVaccinationsLastWeek -->" in line:
@ -162,6 +131,51 @@ def download():
    }


+
+# Function to get data
+def download():
+
+    # Download from open data
+    r = requests.get(DATA_URL)
+    # Create dataframe from data
+    df = pd.read_csv(
+        io.StringIO(r.text),
+        index_col="data_somministrazione",
+    )
+    # Set date as index 
+    df.index = pd.to_datetime(
+        df.index,
+        format="%Y-%m-%d",
+    )
+    # Sort value based on date
+    df.sort_values('data_somministrazione')
+    # Delete sum data if already exists
+    df = df[df["area"] != "ITA"]
+    # Set target counters to numeric
+    df["prima_dose"] = pd.to_numeric(df["prima_dose"])
+    df["seconda_dose"] = pd.to_numeric(df["seconda_dose"])
+    # Group by day and sum counters
+    df_first = df.groupby(['data_somministrazione'])['prima_dose'].sum().reset_index()
+    df_second = df.groupby(['data_somministrazione'])['seconda_dose'].sum().reset_index()
+    # Re-set date as ID in new dataframe
+    df_first = df_first.set_index('data_somministrazione')
+    df_second = df_second.set_index('data_somministrazione')
+    
+    # If there are current day data...
+    if dt.now() - df_first.index[-1] < td(days=1):
+        df_first = df_first[:-1]  # Ignore the current day because it's often incomplete
+    if dt.now() - df_second.index[-1] < td(days=1):
+        df_second = df_second[:-1]  # Ignore the current day because it's often incomplete
+
+    # Generata images
+    intro = generate(df_second, 'seconda_dose', 'assets/templates/intro.html')
+    plot1 = generate(df_first, 'prima_dose', 'assets/templates/plot.html')
+    plot2 = generate(df_second, 'seconda_dose', 'assets/templates/plot.html')
+
+    return intro, plot1, plot2
+
+
+
 # Help command
 def help(update, context):

@ -179,7 +193,6 @@ def help(update, context):



-
 # Info command
 def info(update, context):

@ -193,6 +206,7 @@ def info(update, context):
    context.bot.send_message(chat_id=update.effective_chat.id, text=info_msg, parse_mode='Markdown', disable_web_page_preview=True)


+
 # Start command
 def start(update, context):
    global db
@ -225,6 +239,7 @@ def start(update, context):
    info(update, context)


+
 # Stop command
 def stop(update, context):
    global db
@ -247,6 +262,8 @@ def stop(update, context):
    # Send welcome message
    context.bot.send_message(chat_id=update.effective_chat.id, text=stop_msg, parse_mode='Markdown')

+
+
 # News command
 def news(update, context):
    global db
@ -287,19 +304,30 @@ def news(update, context):
    # Send welcome message
    context.bot.send_message(chat_id=update.effective_chat.id, text=set_msg, parse_mode='Markdown')

+
+
 # Get data
 def get(update, context):

    # Download data
-    data = download()
+    intro, plot1, plot2 = download()
    
-    # Send photo
-    context.bot.send_photo(chat_id=update.effective_chat.id, photo=open(data['results'], 'rb'), caption="")
+    # Send photos
+    p1 = open(intro['results'], 'rb')
+    p2 = open(plot1['results'], 'rb')
+    p3 = open(plot2['results'], 'rb')
+
+    p1 = InputMediaPhoto(media=p1)
+    p2 = InputMediaPhoto(media=p2)
+    p3 = InputMediaPhoto(media=p3)
+
+    context.bot.send_media_group(chat_id=update.effective_chat.id, media=[p1, p2, p3])

    # Remove tmp files
-    os.remove(data['plot'])
-    os.remove(data['webpage'])
-    os.remove(data['results'])
+    for data in [intro, plot1, plot2]:
+        os.remove(data['plot'])
+        os.remove(data['webpage'])
+        os.remove(data['results'])
    


@ -311,7 +339,7 @@ def job():
    print('Job running...')

    # Download data
-    data = download()
+    intro, plot1, plot2 = download()
    
    # Get active user with daily news
    users = db.users.find({
@ -326,8 +354,16 @@ def job():
    for user in users:
        
        try:
-            # Send photo
-            updater.bot.send_photo(chat_id=user['_id'], photo=open(data['results'], 'rb'), caption="")
+            # Send photos
+            p1 = open(intro['results'], 'rb')
+            p2 = open(plot1['results'], 'rb')
+            p3 = open(plot2['results'], 'rb')
+
+            p1 = InputMediaPhoto(media=p1)
+            p2 = InputMediaPhoto(media=p2)
+            p3 = InputMediaPhoto(media=p3)
+
+            updater.bot.send_media_group(chat_id=user['_id'], media=[p1, p2, p3])            
            # Count news successfully sent
            news_sent_counter = news_sent_counter + 1
            # Log
@ -342,9 +378,10 @@ def job():
    print("[JOB] Results: {sent} messages sent, {fail} messages failed.".format(sent=news_sent_counter, fail=news_fail_counter))

    # Remove tmp files
-    os.remove(data['plot'])
-    os.remove(data['webpage'])
-    os.remove(data['results'])
+    for data in [intro, plot1, plot2]:
+        os.remove(data['plot'])
+        os.remove(data['webpage'])
+        os.remove(data['results'])
    


@ -373,7 +410,7 @@ start_handler = CommandHandler('news', news)
 dispatcher.add_handler(start_handler)

 # Setup cron
-schedule.every().day.at("18:00").do(job)
+schedule.every().day.at("18:30").do(job)

 # Start bot
 updater.start_polling()
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -1,14 +1,15 @@
-version: "3.5"
+version: '3.7'

 networks:
  itavscovidbot.network:
    name: itavscovidbot.network
+    driver: bridge


 services:

  db:
-    image: mongo
+    image: mongo:latest
    container_name: itavscovidbot.db
    restart: unless-stopped
    env_file:
@ -21,11 +22,9 @@ services:
  db.web:
    depends_on:
      - db
-    image: mongo-express
+    image: mongo-express:latest
    container_name: itavscovidbot.db.web
    restart: unless-stopped
-    links:
-      - db
    env_file:
      - ./db/web/.env
    ports:
@ -36,13 +35,10 @@ services:
  app:
    depends_on:
      - db
-      - db.web
    build:
      context: .
    container_name: itavscovidbot.app
    restart: unless-stopped
-    links:
-      - db
    env_file:
      - ./.env
    networks:
Author	SHA1	Message	Date
Gabriele De Rosa	0ac692622d	Merge pull request #3 from derogab/dev Fix cron worker	2021-05-21 18:25:03 +02:00
Gabriele De Rosa	0c345f30e9	Change cron hour	2021-05-21 18:24:41 +02:00
Gabriele De Rosa	b195abf771	Fix cron worker	2021-05-21 18:21:32 +02:00
Gabriele De Rosa	bfc8ff46b8	Merge pull request #2 from derogab/dev Minor Improvements	2021-05-21 15:57:35 +02:00
Gabriele De Rosa	07b8aa59f8	Improve intro	2021-05-21 15:55:23 +02:00
Gabriele De Rosa	e74e671ff6	Remove progress bars	2021-05-21 15:51:10 +02:00
Gabriele De Rosa	69d9d57832	Merge pull request #1 from derogab/dev Restyle	2021-05-21 15:19:16 +02:00
Gabriele De Rosa	6ea9453626	Minor improvements	2021-05-21 15:08:13 +02:00
Gabriele De Rosa	d6a93dfd91	Update README.md	2021-05-21 14:50:57 +02:00
Gabriele De Rosa	e8a11e9d42	Update README.md	2021-05-21 14:49:56 +02:00
Gabriele De Rosa	0a5038306e	Total --> First	2021-05-21 14:25:23 +02:00
Gabriele De Rosa	ee5df0c4c7	Restyle	2021-05-21 14:15:34 +02:00
Gabriele De Rosa	374d90b8e3	Minor improvements	2021-05-21 03:28:16 +02:00