SMTP Authentication
[1/10] from: ml:sproingle at: 28-Apr-2004 21:50
My SMTP server requires authentication with a username and password before allowing mail
to be sent.
I found esend.r and esmtp.r, installed those, ran them individually as in the instructions
and then tries to use esend to send a mail.
It ran without an error, but never popped up anuthing to ask me for a username and password
to authenticate with.
Anyone know what the problem is?
Stuart
[2/10] from: antonr:lexicon at: 29-Apr-2004 14:53
I presume the email was actually sent ?
(Send one to yourself to verify.)
I would put some PROBEs into the various
functions of esmtp.r to see what it is
doing. Don't be frightened; they're only functions.
Did you get esmtp.r and esend.r from rebol.org ?
Finally, are you sure the smtp server really is
a Microsoft Exchange Server ?
Anton.
[3/10] from: Gary:Jones:usap:gov at: 29-Apr-2004 17:28
Hi, Stuart,
> My SMTP server requires authentication with a username and
> password before allowing mail to be sent.
<<quoted lines omitted: 4>>
> ask me for a username and password to authenticate with.
>Anyone know what the problem is?
I wrote those patches several years ago. I'm surprised that it ran without error, because
more recent REBOLs less aggressively reduce certain information.
With the newer versions of REBOL, I have successfully used this newer patch with MS Exchange
server(watch for line wrap):
system/schemes/smtp/handler/open-check: [none "220" ["EHLO" system/network/host] "250"
"AUTH LOGIN" "334" [port/user] "334" [port/pass] "235"]
Then, I would throw the following view script in front of any script that used Exchange
email. Note that the script encodes the username and password in base 64, which is required
by MS Exchange Server. You may of course replace "MSusername" with the default that you
would like presented. Hitting return key advances the field and does final entry.
view lo: layout [
text "username"
f: field "MSusername" [
system/schemes/smtp/user: copy enbase/base value 64
]
text "password"
field hide [
system/schemes/smtp/pass: copy enbase/base value 64
unview/all
]
do [focus f]
]
Then, you should be able to use the regular REBOL send command for email.
Without knowing which email smtp server to which you are connecting, it is impossible
to know how to encode the authentication information. I would try this method first,
especially if it is an MS Exchange server. If it does not work, next try the:
trace/net on
command and then send a test email and capture the output. Generally, right before the
authentication is sent the server indicates what form of authentication and encoding
may be required. Obscuring any sensitive information, you can report that back here,
and I or someone may be able to direct you how to change the patch. If that fails to
capture the information, there is yet another way to figure this out that I can explain
more fully.
Best wishes
--Scott Jones
[4/10] from: ml:sproingle at: 29-Apr-2004 15:19
No the message was never received.
And no the SMTP server is not an Exchange box. If this doesn't work with non-Exchange
SMTP servers, do you know of anything that does?
Stuart
[5/10] from: ml::sproingle::com at: 29-Apr-2004 17:34
Thanks Scott.
Sorry I didn't understand that the scripts were for SMTP on MS Exchange only. I know
my ISP is not using Exchange servers, so do you think the scripts should still work,
or is there another way to pass username/password to non Exchange SMTP servers?
Thanks for the help.
Stuart
[6/10] from: Gary:Jones:usap:gov at: 30-Apr-2004 10:56
From: ML
> ...
> Sorry I didn't understand that the scripts were for SMTP
> on MS Exchange only. I know my ISP is not using Exchange
> servers, so do you think the scripts should still work,
> or is there another way to pass username/password to non
> Exchange SMTP servers?
....
Hi, Stuart,
It depends on whether your SMTP server follows the numeric prompt "334" and whether your
SMTP server allows plain text authentication or some other method of encoding. This
was why I suggested turning "trace/net on" and seeing what the net log says the server
is asking.
In the meantime you can still try the patch (watch for line wrap):
system/schemes/smtp/handler/open-check: [none "220" ["EHLO" system/network/host] "250"
"AUTH LOGIN" "334" [port/user] "334" [port/pass] "235"]
Then try to have the user name and password sent as plain (unencoded) text. Use this
script fragment as one way to set these data:
view lo: layout [
text "username"
f: field "username" [
system/schemes/smtp/user: copy value
]
text "password"
field hide [
system/schemes/smtp/pass: copy value
unview/all
]
do [focus f]
]
or alternatively in noninteractive script method:
system/schemes/smtp/user: "myusername"
system/schemes/smtp/pass: "mypassword"
This fragment sets "system/schemes/smtp/user" to your username, and "system/schemes/smtp/pass"
to your password. Within the SMTP scheme "system/schemes/smtp/user" is represented by
the internal path "port/user" and "system/schemes/smtp/pass" is represented by the internal
path "port/pass". So when the SMTP request is being handled and the numeric indicator
"334" is detected, the protocol sends first the port/user, and then the port/pass is
sent for the second "334".
If this suggestion does not work, then try at a *fresh* REBOL session turn the "trace/net
on". Then send a test email and make a copy of the net-log, obscuring any sensitive
information and paste-ing that to the forum. If I/we cannot figure it out from that,
there is one other way (which I'll avoid explaining from now since I do not believe it
will be necessary).
I hope the plain text authentication works. If not let me/us know what the net log shows
from a fresh REBOL session.
--Scott Jones
[7/10] from: ml:sproingle at: 5-May-2004 5:37
Scott,
I hope I followed your directions properly. Can you help me figure out what is wrong?
I created a program with the following contents (username, password and destination address
obscured).
======================================
REBOL []
system/schemes/smtp/handler/open-check: [none "220" ["EHLO" system/network/host] "250"
"AUTH LOGIN" "334"
[port/user] "334" [port/pass] "235"]
system/schemes/smtp/user: "<obscured>"
system/schemes/smtp/pass: "<obscured>"
send [obscured--email--com] "Rebol mail test"
=======================================
I ran this with trace/net on and got the following response (some host names and IP addresses
obscured):
>> do %mailer.r
Script: "Untitled" (none)
Net-log: ["Opening" "tcp" "for" "smtp"]
connecting to: <obscured>
Net-log: [none "220"]
Net-log: {220-<obscured> ESMTP Exim 4.24 #1 Wed, 05 May 2004 02:28:10 -0700 }
Net-log: {220-We do not authorize the use of this system to transport unsolicited, }
Net-log: "220 and/or bulk e-mail."
Net-log: [["EHLO" system/network/host] "250"]
Net-log: {250-<obscured> Hello <obscured> [<obscured>]}
Net-log: "250-SIZE 52428800"
Net-log: "250-PIPELINING"
Net-log: "250-AUTH PLAIN LOGIN"
Net-log: "250-STARTTLS"
Net-log: "250 HELP"
Net-log: ["AUTH LOGIN" "334"]
Net-log: "334 <obscured>"
Net-log: [[port/user] "334"]
** User Error: Server error: tcp 501 Invalid base64 data
** Near: smtp-port: open [scheme: 'smtp]
either only
[8/10] from: Gary:Jones:usap:gov at: 6-May-2004 10:14
Hi, Stuart,
From: Stuart
....
> I hope I followed your directions properly.
> Can you help me figure out what is wrong?
....
for rest of email, see:
http://www.rebol.net/list/list-msgs/35906.html
This is an interesting situation. It appears as though you
did everything correctly, assuming that "<obscured>" was
substituted for "myusername", etc (meaning the angle
brackets were merely placed to help me see the substitution).
The traced output is very helpful, and yet I won't fully be
able to do more research until our satellite that allows
web access comes back up (hours from now).
In the meantime, here is what I see (this email will go out
on our "slow" satellite):
The line:
Net-log: "250-AUTH PLAIN LOGIN"
suggests that a plain (clear) text login is appropriate for
this email server. This is what you have done by manually
setting the username and password.
However, later, the line:
** User Error: Server error: tcp 501 Invalid base64 data
** Near: smtp-port: open [scheme: 'smtp]
would normally suggest that the server was expecting a base
64 encoded string. You tried that last week, but you could
try it again "to be sure". Do exactly what you did but use
a direct encoding like as follows:
system/schemes/smtp/user: enbase/base "myusername" 64
system/schemes/smtp/pass: enbase/base "mypassword" 64
My guess is that this too will fail, like it apparently did
last week. Do it with the trace/net on to see if the message
is any different. My guess is that it will not be
significantly different.
I suspect that the major clue provided is with this line:
Net-log: "250-STARTTLS"
This is the most interesting point that I would like to
research. I vaguely recall that TLS is an encoding or
encrypting scheme. Does anyone else know? If it is then
the email server may be looking for a "plain text" version
of the encoded or encrypted username and password. I do
not recall how this can be done easily through REBOL. Does
anyone else know?
If no one else (that reads this email) has knowledge on
TLS, I would start by googling (meaning, use google.com to
search) the exact string "250-STARTTLS"
Depending on what that yielded, I might next find out if
TLS is an encoding or encrypting scheme.
Depending on what that yielded, I would next find out more
about the email server by googling "ESMTP Exim 4.24".
Sometimes if an email server is not following a standard, there
will be 5 billion references on the net where people have
the same problem, and you can interpolate a REBOL solution
from the help these people receive.
Finally, as a cross check, I would google the line
tcp 501 Invalid base64 data
to be sure that this may not have
an additional meaning besides what may be obvious.
If it ends up being that TLS is an encoding or encrypting
scheme, and if this method is not directly or easily supported
in REBOL, then depending on the method used, you may be able
to do the encoding or encrypting *once* through some other utility
and then plug the results in directly into your scripts.
This is the best I can do without being able to access the web.
If I have time this evening, I will try to look. In the
meantime, I am hoping that either you or someone knows what
these clues mean, and can fill in the gap.
Hang in there. You may get your REBOL solution yet.
--Scott Jones
[9/10] from: g:santilli:tiscalinet:it at: 6-May-2004 12:39
Hi Scott,
On Thursday, May 6, 2004, 12:14:23 AM, you wrote:
JS> I suspect that the major clue provided is with this line:
JS> Net-log: "250-STARTTLS"
JS> This is the most interesting point that I would like to
JS> research. I vaguely recall that TLS is an encoding or
JS> encrypting scheme. Does anyone else know? If it is then
JS> the email server may be looking for a "plain text" version
JS> of the encoded or encrypted username and password. I do
JS> not recall how this can be done easily through REBOL. Does
JS> anyone else know?
TLS is the Transport Layer Security protocol.
http://www.ietf.org/rfc/rfc2246.txt
The TLS protocol provides communications privacy over
the Internet. The protocol allows client/server applications to
communicate in a way that is designed to prevent eavesdropping,
tampering, or message forgery.
TLS and SSL are both supported by REBOL/Command. If your SMTP
server requires a TLS connection, you are out of luck with just
/Core or /View, unless you try to use tools like stunnel etc.
Regards,
Gabriele.
--
Gabriele Santilli <[g--santilli--tiscalinet--it]> -- REBOL Programmer
Amiga Group Italia sez. L'Aquila --- SOON: http://www.rebol.it/
[10/10] from: Gary:Jones:usap:gov at: 7-May-2004 9:10
From: Gabriele Santilli
SJ> I vaguely recall that TLS is an encoding or
SJ> encrypting scheme. Does anyone else know?
....
GS> TLS is the Transport Layer Security protocol.
GS>
GS> http://www.ietf.org/rfc/rfc2246.txt
Thanks for confirming that, Gabriele. I also found:
ftp://ftp.isi.edu/in-notes/rfc2487.txt
that explains more of the overview of smtp server behavior.
I did not have much time to explore the exim server information
on-line, but the information in the second RFC makes me wonder
whether there may not be an alternative for your server. Again,
the server seems to "advertise" that it accepts plain
authentication. As I recall 3ish years ago, the extended smtp
(ESMTP) "standard" is not very standard at all, which was
reportedly why RT decided not to support it directly. Indeed,
over the last several years we've seen several variations on
the list. If you are interested in pursuing this with a non-
command REBOL, I would next communicate with your smtp server
through an interactive interface, most easily done through a
telnet utility.
SMTP servers will detect whether the client is on its net or not,
and can require different logins based on this information. Or
it can simply exclude connections from outside its network. The
2nd RFC implies that if the smtp server is "publically exposed",
then it should not offer services on port 25 (the "usual" smtp
port) *if secured authentication is mandatory*. A different port
(like 995, if I recall) is usually required. This method allows
secured and authenticated transmission of emails from a client, but
is *not* used as a relay mail transport.
Given that your email makes no reference to using any other port
than the default, namely 25, then I am assuming that either the
server is only accessible from within your network (subnet really)
or that it will allow an alternative login. Otherwise, if my
logic holds up, the server would be "noncompliant". As I hinted,
I wonder if the server may allow a plain authenticated login but
through a somewhat nonstandard sequence of commands. Hence the
need to do some testing before deciding that it can't be done
(easily) through "regular" REBOL.
Unfortunately, as I have made indirect reference to several times,
*describing* how to do this is not easy through an asynchronous
method like email. When I helped Nick Carson out a few years ago
he gave me a temporary account on his server, so I could play
with the methods.
Before starting telnet, find out your client name at a utility
of your choice or with REBOL:
probe system/network/host
>> probe system/network/host
NSF000000
At any rate, use a telnet utility to communicate with the server
for an interactive session. From the command line, it might
look something like:
telnet smtp.mydomain.com 25
or once in a telnet session it will more likely look like:
open smtp.mydomain.com 25
It the following the server lines are marked by "S>" and the
client lines are marked by "C>" (meaning, what you type in). Lines
will be wrapped on this email and will of course look different
with your server. Some telnet utilities do not echo your input;
type especially carefully in those cases, or get a telnet utility
that does echo.
S> 220 apricot.southpole.usap.gov Microsoft ESMTP MAIL Service,
Version: 5.0.2195.6 713 ready at Fri, 7 May 2004 08:08:56 +1200
C> HELO NSF000000
S> 250 apricot.southpole.usap.gov Hello [157.132.6.107]
C> HELP
S> 214-This server supports the following commands:
S> 214 HELO EHLO STARTTLS RCPT DATA RSET MAIL QUIT HELP AUTH TURN
ATRN ETRN BDAT VRFY
C> QUIT
S> 221 2.0.0 apricot.southpole.usap.gov Service closing
transmission channel
In this simple sequence we have established basic communications with
the smtp server. HELO is a standard log greeting typically used when
NO authentication is required. The HELP command speaks for itself, and
will tell us what general commands are allowed. Notice the EHLO command.
EHLO was added with the extended specification as a way of offering the
servers a way to allow logins with various levels of authentication.
You will notice that the patch I suggested substitutes this log greeting
for HELO.
If this first session does not go well, it may be helpful to cut and
paste the information for the list to see. Assuming the *does* go well,
next try the following at a telnet session:
S> 220 apricot.southpole.usap.gov Microsoft ESMTP MAIL Service,
Version: 5.0.2195.6 713 ready at Fri, 7 May 2004 08:08:56 +1200
C> HELO NSF000000
S> 250 apricot.southpole.usap.gov Hello [157.132.6.107]
C> AUTH PLAIN
....
(*BookMark 1*)
One of two things will likely happen. The first will tell me that
the server is quirky in its authentication method (not following the
standard directly). I can't tell you exactly what to look for but it
will likely either be a 334 code followed by either the text "Username:"
or the encoded text "VXNlcm5hbWU6" (which is "Username:" in base64).
It is possible that it may give a different code.
If you get something along these lines, then if the text prompt is
Username:
then type your smtp service username and hit enter (return).
Next you should see "334 Password:", then type your password and enter.
If you see:
S> 235 2.7.0 Authentication successful.
Then you are in! You may type "QUIT" to exit. If it is not obvious
how to change my patch, let me know, and I hopefully can change the
stimulus/response
pairs to get you logged.
If however you see "334 VXNlcm5hbWU6", then you will need to supply
a base 64 encoded username. At REBOL type:
enbase/base "myusername" 64
at a REBOL session. Take the result, excluding the double quotes,
and paste it into the telnet session and hit enter.
Next you will see "334 UGFzc3dvcmQ6", so you will need to supply
the base 64 encoded password just like for the username. If you see:
s> 235 2.7.0 Authentication successful.
Then you are in! You may type "QUIT" to exit. If it is not obvious
how to change my patch, let me know, and I hopefully can change the
stimulus/response
pairs to get you logged.
However, more likely you will see something like this 503 error after
typing "AUTH PLAIN" for a "HELO" greeting:
S> 503 5.5.2 Send hello first.
Then that will tell me that the server is *not* totally quirky in its
authentication method. Next, either in the same session or a new
session, type:
C> EHLO NSF000000
To be explicitly clear, note the HELO is scrambled to EHLO. This *is*
supposed to designate that a form of authenticated login is to be used.
Following this, based on your earlier email, *you* will see this:
S> 250-<obscured> Hello <obscured> [<obscured>]
S> 250-SIZE 52428800
S> 250-PIPELINING
S> 250-AUTH PLAIN LOGIN
S> 250-STARTTLS
S> 250 HELP
Through the patch, we've already tried "AUTH LOGIN", which seems to
possibly require TLS (we aren't totally sure based on the information).
This time try "AUTH PLAIN". Again, one of two things may happen.
Return to "(*BookMark 1*)" in this email and try the same approach.
My hope and intuition suggests that you may be able to do a truly
plain
authentication login at that juncture.
As far as other tips, keep an eye out for non-standard codes. The REBOL
scheme is absolutely dependent on finding these codes. The last code
in a sequence *should* have a space following before the text. The
only other quirk/tip is that some services require the full email
address for the username. You will likely already know about this
from configuring your other email clients.
I hope we may stumble across the right solution. It helps you and
others and improves REBOL's future for use. Please, no matter what
you find positive or negative, let us/me know so that we have a better
leg up
for the next person!
Thanks!
--Scott Jones
Weather for South Pole Station
The date is 05-07-2004 at 9:00 AM
Temperature -63.3 C -82 F
Windchill -100.1 C -148.2 F
Wind 14.7 kts Grid 042
Barometer 670.3 mb (10993. ft)
UTC 05-06-2004 at 21:00 Z
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted