Send SMS clickatell helpers

[DewGew]

I have an issue with characters. I usually use UTF-8 when I create my scripts. But when I call this helpers I have to use ANSI to get correct characters to send sms. I'm swedish and we use å ä ö. Now my issue is that if I need to put something in the log I need to have UTF-8 else I get "Inbrotts larmet har utl�sts! Anledningen �r"
How can I get around this?

my script:

Code: Select all

return {
	active = true,
		on = {
		devices = {
   			'PIR*',
			}
		},

	execute = function(domoticz, device)
	
			local meddelande = 'Inbrotts larmet har utlösts! Anledningen är:'
		
		if (device.state == 'On') then
			domoticz.helpers.sendClickatell(domoticz, meddelande, 'Pierre')
			domoticz.log(meddelande, domoticz.LOG_ERROR)
		end
	end
}

Helper script:

Code: Select all

local phoneNumbers = {
['PersonA'] = '+31xxxxx',
['PersonB'] = '+31yyyyy'
}

local function smsEncode(str)
-- Convert to GSM 03.38 character set and URL-encode
local utf8Chars={"%%","\n"," ",'"',"&",",","%.","/",":",";","<","=",">","?","¡","£","#","¥","§","Ä","Å","à","ä","å","Æ","Ç","É","è","é","ì","Ñ","ñ","ò","ö","Ø","Ö","Ü","ù","ü","ß", "\\", "*", "'", "%(", "%)", "@", "%+", "%$", "%[", "%]", "%^", "{", "|", "}", "~"}
local gsmChars={"%%25","%%0D","%%20","%%22","%%26","%%2C","%%2E","%%2F","%%3A","%%3B","%%3C","%%3D","%%3E","%%3F","%%A1","%%A3","%%A4","%%A5","%%A7","%%C4","%%C5","%%E0","%%E4","%%E5","%%C6","%%C7","%%C9","%%E8","%%E9","%%EC","%%D1","%%F1","%%F2","%%F6","%%D8","%%D6","%%DC","%%F9","%%FC","%%DF","%%5C","%%2A","%%27","%%28","%%29","%%40","%%2B","%%24","%%5B","%%5D","%%5E","%%7B","%%7C","%%7D","%%7E"}
for i = 1, #utf8Chars, 1 do
str = string.gsub(str, utf8Chars, gsmChars)
end
return str 
end

return {
helpers = {

sendClickatell = function(domoticz, message, user, phoneNumber)

local APIId = "<your clickatell API code"
local encMsg = smsEncode(message)
local number = phoneNumber -- optional

if (phoneNumber == nil) then
-- get it from the list above
number = phoneNumbers[user] 
end

local url = "https://platform.clickatell.com/message ... nd?apiKey=" .. 
APIId .. 
"&to=" .. 
number .. 
"&content=" ..
encMsg

domoticz.openURL(url)
end
}
}

[BakSeeDaa]

The meddelande variable is a string and should be passed by value, not by reference so it's kind of confusing for me that it changes when you call the sendClickatell function. Anyway, try to make the logging before calling sendClickatell. Does it make any difference?

[BakSeeDaa]

...
How can I get around this?
...

Hi again. It seems impossible that the code you've shown us above will output

Inbrotts larmet har utl�sts! Anledningen �r

What we see is actually the smsEncoded version of the value kept in the variable meddelande.

To see that string in the log, with that wrong characters you must have printed it out (maybe using domoticz.log) somewhere in the helper script. Your main script just can't do that. It's because the function that you call sendClickatell, can not change the value of your local variable meddelande. Whatever is inside the variable named meddelande is sent "by value". So I believe you must have changed the code in some way before showing it here for us.

By the way, there is problem with the code that you've quoted from the function smsEncode. (I actually wrote that piece of code some time ago).

Code: Select all

for i = 1, #utf8Chars, 1 do
	str = string.gsub(str, utf8Chars, gsmChars)
end

is wrong.

It should be

Code: Select all

for i = 1, #utf8Chars, 1 do
	str = string.gsub(str, utf8Chars[i], gsmChars[i])
end

So if you just change that and nothing else, I'm quite sure about that your output in the log will be

Code: Select all

Inbrotts larmet har utlösts! Anledningen är:

Tjolahopp!

[DewGew]

The meddelande variable is a string and should be passed by value, not by reference so it's kind of confusing for me that it changes when you call the sendClickatell function. Anyway, try to make the logging before calling sendClickatell. Does it make any difference?

I tryed to change the code in my global_data.lua as you suggested, but its the same problem. My script (not global_data.lua) that calls the helper needs to be in ANSI, if the script is UTF-8 my script send the SMS "Inbrotts larmet har utl�sts! Anledningen �r:" ??

So it looks like this:
global_data_lua - UTF-8
myscript.lua with ANSI - Send SMS right char, log wrong char.
myscript.lua with UTF-8 Send SMS wrong char, log right char.

[BakSeeDaa]

I tryed to change the code in my global_data.lua as you suggested, but its the same problem. My script (not global_data.lua) that calls the helper needs to be in ANSI, if the script is UTF-8 my script send the SMS "Inbrotts larmet har utl�sts! Anledningen �r:" ??

So it looks like this:
global_data_lua - UTF-8
myscript.lua with ANSI - Send SMS right char, log wrong char.
myscript.lua with UTF-8 Send SMS wrong char, log right char.

Make both your scripts UTF-8 and save them. After doing that please paste your scripts here again and I shall have a look.

Below are my scripts that works well on my system.
testa.lua

Code: Select all

return {
	active = true,
	logging = {
		level = domoticz.LOG_INFO,
		marker = "TEST"
	},
	on = {
		devices = {
			'Test Button',
		},
	},
	execute = function(domoticz, device)
		local msg = 'åäöÅÄÖ'
		
		if device.name == 'Test Button' and device.state == 'On' then
			domoticz.log('Sending an SMS with the message: '..msg, domoticz.LOG_INFO)
			domoticz.helpers.sendSMS(domoticz, msg)
		end

	end
}

function from global_data.lua

Code: Select all

		-- SMS messages are sent by making HTTP calls through the Clickatell API
		-- The optional argument ''subscriber'' can be a named person or a phone number 
		sendSMS = function(domoticz, message, subscriber)
			local ClickatellSender = '46701111111'
			local ClickatellAPIUser = 'xxxxxxxxxx'
			local ClickatellAPIPassw = 'xxxxxxxxxx
			local ClickatellAPIId = 999999999
			local phoneNumbers = {
			    ['Kalle'] = '46701111111',
			    ['Nisse'] = '46731111111',
			}
			subscriber = subscriber or 'Kalle' -- default

			local function smsEncode(str)
				-- Convert to GSM 03.38 character set and URL-encode
				local utf8Chars={'%%','\n',' ','"','&',',','%.','/',':',';','<','=','>','?','¡','£','#','¥','§','Ä','Å','à','ä','å','Æ','Ç','É','è','é','ì','Ñ','ñ','ò','ö','Ø','Ö','Ü','ù','ü','ß', '\\', '*', '\'', '%(', '%)', '@', '%+', '%$', '%[', '%]', '%^', '{', '|',  '}', '~'}
				local gsmChars={'%%25','%%0D','%%20','%%22','%%26','%%2C','%%2E','%%2F','%%3A','%%3B','%%3C','%%3D','%%3E','%%3F','%%A1','%%A3','%%A4','%%A5','%%A7','%%C4','%%C5','%%E0','%%E4','%%E5','%%C6','%%C7','%%C9','%%E8','%%E9','%%EC','%%D1','%%F1','%%F2','%%F6','%%D8','%%D6','%%DC','%%F9','%%FC','%%DF','%%5C','%%2A','%%27','%%28','%%29','%%40','%%2B','%%24','%%5B','%%5D','%%5E','%%7B','%%7C','%%7D','%%7E'}
				for i = 1, #utf8Chars, 1 do
					str = string.gsub(str, utf8Chars[i], gsmChars[i])
				end
				return str 
			end

			local phoneNumber
			if phoneNumbers[subscriber] then
				phoneNumber = phoneNumbers[subscriber]
			elseif tonumber(subscriber) ~= nil then
				phoneNumber = subscriber
			end
			if phoneNumber ~= nil then
				domoticz.log('Sending out an SMS to '..subscriber, domoticz.LOG_INFO)
				domoticz.openURL('https://api.clickatell.com/http/sendmsg?user='..ClickatellAPIUser..'&password='..ClickatellAPIPassw..'&api_id='..ClickatellAPIId..'&from=+'..ClickatellSender..'&to='..phoneNumber..'&text='.. smsEncode(message))
			else
				domoticz.log('sendSMS: Subscriber is unknown: '..subscriber, domoticz.LOG_ERROR)
			end
		end,

[DewGew]

Make both your scripts UTF-8 and save them. After doing that please paste your scripts here again and I shall have a look.

Now my scripts are both in UTF-8 and my messages look like this when send from my test script:

SMS = "Inbrotts larmet har utl�sts! Anledningen �r:"
Log = "Inbrotts larmet har utlösts! Anledningen är:"
Notify = "Inbrotts larmet har utlösts! Anledningen är:"

My global_data.lua:

Code: Select all

local phoneNumbers = {
	['Person1'] = '+123456789',
    ['Person2'] = '+46123456789'
}

local function smsEncode(str)
      -- Convert to GSM 03.38 character set and URL-encode
      local utf8Chars={"%%","\n"," ",'"',"&",",","%.","/",":",";","<","=",">","?","¡","£","#","¥","§","Ä","Å","à","ä","å","Æ","Ç","É","è","é","ì","Ñ","ñ","ò","ö","Ø","Ö","Ü","ù","ü","ß", "\\", "*", "'", "%(", "%)", "@", "%+", "%$", "%[", "%]", "%^", "{", "|",  "}", "~"}
      local gsmChars={"%%25","%%0D","%%20","%%22","%%26","%%2C","%%2E","%%2F","%%3A","%%3B","%%3C","%%3D","%%3E","%%3F","%%A1","%%A3","%%A4","%%A5","%%A7","%%C4","%%C5","%%E0","%%E4","%%E5","%%C6","%%C7","%%C9","%%E8","%%E9","%%EC","%%D1","%%F1","%%F2","%%F6","%%D8","%%D6","%%DC","%%F9","%%FC","%%DF","%%5C","%%2A","%%27","%%28","%%29","%%40","%%2B","%%24","%%5B","%%5D","%%5E","%%7B","%%7C","%%7D","%%7E"}
      for i = 1, #utf8Chars, 1 do
         str = string.gsub(str, utf8Chars[i], gsmChars[i])
      end
      return str 
end

return {
	helpers = {

			-- Send SMS with Clickatell --
        	sendClickatell = function(domoticz, message, user, phoneNumber)
	            
	            local APIId = "xxxXxxXxxx"
	            local encMsg = smsEncode(message)
	            local number = phoneNumber
	            
	            if (phoneNumber == nil) then
	            	-- get it from the list above
	                number = phoneNumbers[user] 
	            end
	            
	            local url = "https://platform.clickatell.com/messages/http/send?apiKey=" .. APIId .. "&to=" .. number .. "&content=" .. encMsg
				domoticz.log('Skickar SMS: ' .. message, domoticz.LOG_FORCE)
	            domoticz.openURL(url)
	        end,
					
			-- Text to Speach (izsynth) --
			speak = function(domoticz, msg)
				
				domoticz.log('Säger: '.. msg, domoticz.LOG_FORCE)
				os.execute("sudo killall mplayer")
				os.execute("sh /home/pi/domoticz/scripts/bash/Play_sound.sh '" .. msg .. "' ")
			end
	}
}

Mys test.lua::

Code: Select all

return {
	
	active = true,

		on = {
		devices = {
		    
			'testaSMS', -- device name
		
			}

		},

	execute = function(domoticz, device)
	
			local meddelande = 'Inbrotts larmet har utlösts! Anledningen är:'
		
		if (device.state == 'On') then
			domoticz.helpers.sendClickatell(domoticz, meddelande, 'Person1')
			domoticz.log(meddelande, domoticz.LOG_FORCE)
			domoticz.notify('Testa åäö',meddelande,domoticz.PRIORITY_EMERGENCY)
		end
	end
}

Edit:
I have a newer account on clickatell and it looks like they have changed something. On the clickatell support pages it says they support UTF-8.

[BakSeeDaa]

Thanks @DewGew

You are using a different API than I do.

I need to use the old API because I created my Clickatell account prior to 2016 and there seems to be no way for me to log in to their system right now.

Anyway, I believe that the API that you are using doesn't require GSM 03.38 encoded messages.

Unfortunately I can not try it out here right now until Clickatell support let's me in so I can get the new type of APIkey.

Can you try to add this function in global_data.lua, just above the smsEncode function

Code: Select all

local function urlEncode(str)
	if (str) then
		str = string.gsub (str, '\n', '\r\n')
		str = string.gsub (str, '([^%w ])',
		function (c) return string.format ('%%%02X', string.byte(c)) end)
		str = string.gsub (str, ' ', '+')
	end
	return str
end

Then make a change.

Replace the code:

Code: Select all

local encMsg = smsEncode(message)

With

Code: Select all

--local encMsg = smsEncode(message)
local encMsg = urlEncode(message)

Then please try again and see if it works!

[BakSeeDaa]

Edit:
I have a newer account on clickatell and it looks like they have changed something. On the clickatell support pages it says they support UTF-8.

Yeahh, I just saw your "edit" now.

[DewGew]

Now I dont recieve any sms at all i get this error:

2017-08-21 11:33:57.317 Error: Error opening url: https://platform.clickatell.com/message ... t=Inbrotts larmet har utlösts! Anledningen är:

The url to send sms look like this:
https://platform.clickatell.com/message ... ssage+text
or
https://platform.clickatell.com/message ... age%20text

to send thrue the browser I can put the 'message' like this "Tästa+message+test" or "Tästa message test" both works.

[BakSeeDaa]

I can't test here until Clickatell gives me support so I can log in to their system.

Can you show me the urlEncoded message?

After

Code: Select all

local encMsg = urlEncode(message)

add:

Code: Select all

domoticz.log('URLEncoded message:  '..encMsg, domoticz.LOG_FORCE)

Thanks

[DewGew]

I can't test here until Clickatell gives me support so I can log in to their system.

Can you show me the urlEncoded message?

After

Code: Select all

local encMsg = urlEncode(message)

add:

Code: Select all

domoticz.log('URLEncoded message:  '..encMsg, domoticz.LOG_FORCE)

Thanks

2017-08-21 12:51:23.182 dzVents: !Info: URLEncoded message: Inbrotts+larmet+har+utl%C3%B6sts%21+Anledningen+%C3%A4r%3A

[DewGew]

IOao !! It works with your last code.. i tryed all over again and now it works

Thanks. Du är en klippa!!