Automatically clean telegram messages

Moderator: leecollings

Post Reply
User avatar
Egregius
Posts: 2582
Joined: Thursday 09 April 2015 12:19
Target OS: Linux
Domoticz version: v2024.7
Location: Beitem, BE
Contact:

Automatically clean telegram messages

Post by Egregius »

Maybe not directly a script for Domoticz, but lot's of us use Telegram for notifications.
When (on iPhone) you receive the messages from the bot as single user you can easily remove the history from the chat, but when using a channel (so multiple users receive the same message) you just can't. Then you have to select each individual message to remove it. A huge work if you use the Telegram system a lot.

Based on the Telegram CLI program I created a script that removes all messages that are older than x seconds. Freeing the chat history and memory of your phone. Especially handy if you also receive pictures from cameras.

First you need to get the Telegram-cli from https://github.com/vysheng/tg up and running.

Then create a file /home/pi/cleantelegram.sh and make it executable

Code: Select all

#!/bin/bash
pgrep telegram-cli && running=1 || running=0
if [ $running == 0 ] ; then
	echo $(/home/pi/tg/bin/telegram-cli -k tg-server.pub -W -N -e 'history Egregius 5000' --json -C --permanent-msg-ids) > /var/log/telegram.txt
	sleep 1
	/home/pi/cleantelegram.php channel 259200
	sleep 1
	echo $(/home/pi/tg/bin/telegram-cli -k tg-server.pub -W -N -e 'history Egregius#1 5000' --json -C --permanent-msg-ids) > /var/log/telegram.txt
	sleep 1
	/home/pi/cleantelegram.php bot 259200
else
	echo 'already running'
	sleep 5
	sudo killall telegram-cli
fi
I am using the bot for technical notifications and a channel for general stuff that my wife is also interested in, that's why I run it twice.

Then create the /home/pi/cleantelegram.php file and make it executable

Code: Select all

#!/usr/bin/php
<?php
error_reporting(E_ALL);ini_set("display_errors","on");
$data=file_get_contents('/var/log/telegram.txt');
$data=strstr($data,"[{");
$data=substr($data,1);
$data = substr($data, 0, strpos($data, "]"));
$datas=explode('}, {',$data);
foreach($datas as $data){
    $encoded='{'.$data.'}';
    $encoded=str_replace('{{','{',$encoded);
    $decoded=json_decode($encoded,false);
    $msg='';
    if(!empty($decoded->id)&&!empty($decoded->date)) {
        //This is the message id from the text 'channel created' that can't be deleted, change it if needed
        if($decoded->id!='05000000980ee83e010000000000000044653f1eedca68e6'){
            if($decoded->date<(time()-$argv[2])) {
                $clean= $decoded->id;$tijd=$decoded->date;
                isset($decoded->text)?$tekst=$decoded->text:$tekst='FOTO';
                $msg.='-'.strftime("%e-%d %k:%M",$tijd).' '.$tekst;
                logwrite($msg,$argv[1]);
                shell_exec("/home/pi/tg/bin/telegram-cli -k tg-server.pub -W -N -e 'delete_msg $clean' -C --permanent-msg-ids");
                print $msg.PHP_EOL;
                sleep(15);
            } else {
                logwrite('Arrived at recent messages');
                die('Arrived at recent messages of '.$argv[1].PHP_EOL);
            }
        }
    }    
}
function logwrite($msg,$msg2=NULL){
    $time=microtime(true);$dFormat="Y-m-d H:i:s";$mSecs=$time-floor($time);$mSecs=substr(number_format($mSecs,3),1);
    $fp=fopen('/files/temp/cleantelegram.txt',"a+");fwrite($fp,sprintf("%s%s %s %s\n",date($dFormat),$mSecs,$msg2,$msg));fclose($fp);
}
 
Add the shell script to cron (I have unlimited internet during the night, so I only run it then)

Code: Select all

0 1,11 * * * /home/pi/cleantelegram.sh
Enjoy!
PeterB1

Re: Automatically clean telegram messages

Post by PeterB1 »

I've been using Telegram for notifications for a while now. I send the messages to a channel. I would also like to delete my messages via an automatic delete. I have quite a few notifications. Now my questions:
- Does Domoticz also work after Telegram CLI program with its packages has been installed?
- I am not an expert on PHP. I then want to install php on my raspberrypi that also runs Domoticz. Which PHP version should I install? Which packages should I install with?
- Does installing PHP still cause problems for Domoticz? Or do I have to run the scripts on a separate RPI?

Already thanks for an answer.
User avatar
sincze
Posts: 1299
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.4
Location: Netherlands / Breda Area
Contact:

Re: Automatically clean telegram messages

Post by sincze »

PeterB1 wrote: Friday 09 October 2020 14:17 I am not an expert on PHP. I then want to install php on my raspberrypi that also runs Domoticz. Which PHP version should I install? Which packages should I install with?
Is all working on 1 Pi ;-) (Domoticz / PHP / Pass2PHP / MariaDB)
I've an RPI 3B with php7.3
Pass2php
LAN: RFLink, P1, OTGW, MySensors
USB: RFXCom, ZWave, Sonoff 3
MQTT: ZIgbee2MQTT,
ZWAVE: Zwave-JS-UI
WIFI: Mi-light, Tasmota, Xiaomi Shelly
Solar: Omnik, PVOutput
Video: Kodi, Harmony HUB, Chromecast
Sensors: You name it I got 1.
User avatar
sincze
Posts: 1299
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.4
Location: Netherlands / Breda Area
Contact:

Re: Automatically clean telegram messages

Post by sincze »

To at least have this solution successfully build on my Pi3B

Code: Select all

Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux
I had to do the following:

Code: Select all

git clone --recursive https://github.com/vysheng/tg.git && cd tg

Code: Select all

sudo apt-get install libreadline-dev libconfig-dev libssl-dev lua5.2 liblua5.2-dev libevent-dev libjansson-dev libpython-dev libgcrypt20-dev make

Code: Select all

./configure --disable-openssl

Code: Select all

make 
Executing

Code: Select all

 ./bin/telegram-cli

Gave me an error “assertion “0” failed”. It seems a known issue.
I decided to comment the line number 101 in the “tgl/mtproto-utils.c” file, compile the binary again, and give it another try.

Original:
Spoiler: show

Code: Select all

static unsigned long long BN2ull (TGLC_bn *b) {
  if (sizeof (unsigned long) == 8) {
    return TGLC_bn_get_word (b);
  } else if (sizeof (unsigned long long) == 8) {
    assert (0); // As long as nobody ever uses this code, assume it is broken.
    unsigned long long tmp;
    /* Here be dragons, but it should be okay due to be64toh */
    TGLC_bn_bn2bin (b, (unsigned char *) &tmp);
    return be64toh (tmp);
  } else {
    assert (0);
  }
}
Modified:
Spoiler: show

Code: Select all

static unsigned long long BN2ull (TGLC_bn *b) {
  if (sizeof (unsigned long) == 8) {
    return TGLC_bn_get_word (b);
  } else if (sizeof (unsigned long long) == 8) {
//    assert (0); // As long as nobody ever uses this code, assume it is broken.
    unsigned long long tmp;
    /* Here be dragons, but it should be okay due to be64toh */
    TGLC_bn_bn2bin (b, (unsigned char *) &tmp);
    return be64toh (tmp);
  } else {
    assert (0);
  }
}
Recompile:

Code: Select all

./configure --disable-openssl

Code: Select all

make 
Execute time:

Code: Select all

sudo ./bin/telegram-cli
 
show_license
 > enter your phone number
 > enter received code

Now it was working.

Modified the original script a bit so it now logs:

Code: Select all

- Removed data from: Mon 05-10-2020 06:30 (Groupname) FOTO
- Removed data from: Mon 05-10-2020 06:30 (Groupname) FOTO
- Removed data from: Mon 05-10-2020 08:54 (Groupname) Voordeur is Geopend
- Removed data from: Mon 05-10-2020 08:54 (Groupname) FOTO
- Removed data from: Mon 05-10-2020 08:54 (Groupname) FOTO
Pass2php
LAN: RFLink, P1, OTGW, MySensors
USB: RFXCom, ZWave, Sonoff 3
MQTT: ZIgbee2MQTT,
ZWAVE: Zwave-JS-UI
WIFI: Mi-light, Tasmota, Xiaomi Shelly
Solar: Omnik, PVOutput
Video: Kodi, Harmony HUB, Chromecast
Sensors: You name it I got 1.
PeterB1

Re: Automatically clean telegram messages

Post by PeterB1 »

Thanks for the comprehensive answer. I was able to install Telegram-cli. I have adapted the enclosed script examples a bit. Only messages are not deleted. What do I specifically need to adjust for myself? My channelname is 'Berichten'.

Do I need to adjust the id mentioned in cleantelegram.php (see below). Is this the channel_id from my channel?

Code: Select all

//This is the message id from the text 'channel created' that can't be deleted, change it if needed
        if($decoded->id!='0500000xxxxxxxxxxxxxxxxxxxxxxxxx44653f1eedca68e6'){

If I type channel_info in telegram-cli I get an error.

Code: Select all

> channel_info Berichten
FAIL: 38: can not parse arg #1
Last edited by PeterB1 on Wednesday 21 October 2020 16:01, edited 1 time in total.
User avatar
sincze
Posts: 1299
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.4
Location: Netherlands / Breda Area
Contact:

Re: Automatically clean telegram messages

Post by sincze »

PeterB1 wrote: Sunday 11 October 2020 10:26 Thanks for the comprehensive answer. I was able to install Telegram-cli. I have adapted the enclosed script examples a bit. Only messages are not deleted. What do I specifically need to adjust for myself? My channelname is 'Berichten'.

Do I need to adjust the id mentioned in cleantelegram.php (see below). Is this the channel_id from my channel?

Code: Select all

//This is the message id from the text 'channel created' that can't be deleted, change it if needed
        if($decoded->id!='05000000980ee83e010000000000000044653f1eedca68e6'){

If I type channel_info in telegram-cli I get an error.

Code: Select all

> channel_info Berichten
FAIL: 38: can not parse arg #1
Ok.
- Please check first if you can send messages and see the incoming messages in telegram-cli. (just send a few).
- If that is working check if the log file is created. Mine is located /var/log/telegram.txt to be able to write here I had to run the script with "sudo" elevated rights.

Code: Select all

sudo ./cleantelegram.sh
- You will find the id for your php in there so you can modify it.

Code: Select all

 {"event": "message", "id": "05000000f***********000000000000***********1a**d", "flags": 256, "from": .......
My .sh looks like

Code: Select all

 echo $(/home/pi/tg/bin/telegram-cli -k /home/pi/tg/tg-server.pub -U root -W -N -e 'history Berichten --json -C --permanent-msg-ids) > /var/log/telegram.txt
Mine is still Purging messages ;-) from 2018 since last night ;-)

Code: Select all

2020-10-11 10:28:49.519 channel - Removed data from: Tue 10-04-2018 13:12 (Berichten) Domoticz: IP-cam niet beschikbaar.
2020-10-11 10:29:05.043 channel - Removed data from: Tue 10-04-2018 13:12 (Berichten) Domoticz: Er is zojuiist aangebeld!
Pass2php
LAN: RFLink, P1, OTGW, MySensors
USB: RFXCom, ZWave, Sonoff 3
MQTT: ZIgbee2MQTT,
ZWAVE: Zwave-JS-UI
WIFI: Mi-light, Tasmota, Xiaomi Shelly
Solar: Omnik, PVOutput
Video: Kodi, Harmony HUB, Chromecast
Sensors: You name it I got 1.
PeterB1

Re: Automatically clean telegram messages

Post by PeterB1 »

Again this helped me. I works. Thanks.

I had three issues which I solved:
- I got a error installing Telegram-cli using the given link. This works.

Code: Select all

git clone --recursive https://github.com/kenorb-contrib/tg.git && cd tg
- File permission issues. Should I use root? I solved this.
- Because I used the app VNCviewer on a Macbook Pro to work with the Raspberrypi the shell script gave errors. I presume hidden mac characters.
User avatar
sincze
Posts: 1299
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.4
Location: Netherlands / Breda Area
Contact:

Re: Automatically clean telegram messages

Post by sincze »

Ah yes sometimes install needs some digging around. :lol:

Changed the .sh file to so it can run without root/sudo rights on my Pi3B.

Code: Select all

#!/bin/bash
file="/var/log/telegram.txt"

if [ ! -f "$file" ]; then
       sudo touch $file
       sudo chown "pi:pi" $file
else
       echo 'File exist all okay continue'
fi

pgrep telegram-cli && running=1 || running=0
if [ $running == 0 ] ; then

        echo $(/home/pi/tg/bin/telegram-cli -k /home/pi/tg/tg-server.pub  -W -N -e 'history Berichten 5000' --json -C --permanent-msg-ids) > $file
        sleep 1
        /home/pi/cleantelegram.php channel  259200
else
        echo 'already running'
        sleep 5
        sudo killall telegram-cli
fi
Pass2php
LAN: RFLink, P1, OTGW, MySensors
USB: RFXCom, ZWave, Sonoff 3
MQTT: ZIgbee2MQTT,
ZWAVE: Zwave-JS-UI
WIFI: Mi-light, Tasmota, Xiaomi Shelly
Solar: Omnik, PVOutput
Video: Kodi, Harmony HUB, Chromecast
Sensors: You name it I got 1.
Dalle
Posts: 2
Joined: Tuesday 20 October 2020 18:38
Target OS: Linux
Domoticz version:
Contact:

Re: Automatically clean telegram messages

Post by Dalle »

Hi,

I did everything as described. Only there is no error message or something happens.
In the text file I see the messages from my channel, I'm just not sure what is behind:

/home/pi/cleantelegram.php channel

heard ? Which ID should that be.


I have an error message. change_user:group: can´t find the user telegramd to switch to
User avatar
sincze
Posts: 1299
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.4
Location: Netherlands / Breda Area
Contact:

Re: Automatically clean telegram messages

Post by sincze »

Dalle wrote: Tuesday 20 October 2020 18:43 Hi,

I did everything as described. Only there is no error message or something happens.
In the text file I see the messages from my channel, I'm just not sure what is behind:

/home/pi/cleantelegram.php channel

heard ? Which ID should that be.


I have an error message. change_user:group: can´t find the user telegramd to switch to
That is the amount of seconds :)

"I have an error message. change_user:group: can´t find the user telegramd to switch to" -> you are running with sudo I presume?
echo $(/home/pi/tg/bin/telegram-cli -k /home/pi/tg/tg-server.pub -U root
Pass2php
LAN: RFLink, P1, OTGW, MySensors
USB: RFXCom, ZWave, Sonoff 3
MQTT: ZIgbee2MQTT,
ZWAVE: Zwave-JS-UI
WIFI: Mi-light, Tasmota, Xiaomi Shelly
Solar: Omnik, PVOutput
Video: Kodi, Harmony HUB, Chromecast
Sensors: You name it I got 1.
Dalle
Posts: 2
Joined: Tuesday 20 October 2020 18:38
Target OS: Linux
Domoticz version:
Contact:

Re: Automatically clean telegram messages

Post by Dalle »

Thanks, I adjusted the time, now you are. But the message still comes.
User avatar
sincze
Posts: 1299
Joined: Monday 02 June 2014 22:46
Target OS: Raspberry Pi / ODroid
Domoticz version: 2024.4
Location: Netherlands / Breda Area
Contact:

Re: Automatically clean telegram messages

Post by sincze »

Please share your command line & script for inspection.
Pass2php
LAN: RFLink, P1, OTGW, MySensors
USB: RFXCom, ZWave, Sonoff 3
MQTT: ZIgbee2MQTT,
ZWAVE: Zwave-JS-UI
WIFI: Mi-light, Tasmota, Xiaomi Shelly
Solar: Omnik, PVOutput
Video: Kodi, Harmony HUB, Chromecast
Sensors: You name it I got 1.
kubrik
Posts: 47
Joined: Wednesday 30 November 2016 15:07
Target OS: Linux
Domoticz version: 2020.2
Location: Italy
Contact:

Re: Automatically clean telegram messages

Post by kubrik »

Hi,
with this, you can also use offset in history for channels:

https://github.com/vysheng/tg/issues/94 ... -186102980
Ubuntu 20.04 on ACEPC AK1 (previously RPi3b+)
Z-Stick Gen5, RFXtrx433E, Philips HUE Bridge, Yeelight, ESP8266 NodeMCU, Broadlink RM
dZvents, bash, php, Node-RED
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest