Amazon S3 Tools linux problem.
[1/21] from: carl:cybercraft at: 20-Sep-2010 20:22
Hi all,
I've got the Amazon S3 tools (see: http://rebol.wik.is/S3 ) to work on Windows XP, but
I get a 403 Forbidden error on Linux when using the Get-s3object function to try and
download a file. (I've not tried uploading from Linux yet.)
Does anyone know what could be causing the difference in behaviour? (Note I'm no Linux
expert.)
-- Carl Read.
[2/21] from: compkarori:gm:ail at: 20-Sep-2010 20:37
Hi Carl
I never tested on Linux ... but the forbidden error suggests that the
authentication is not working.
As an alternative Chris rewrote this as a protocol. Perhaps try that as
well?
http://www.ross-gill.com/r/s3http.r
If you find the problem, let me know so I can fix it :)
On Mon, Sep 20, 2010 at 8:22 PM, Carl Read <carl-cybercraft.co.nz> wrote:
> Hi all,
> I've got the Amazon S3 tools (see: http://rebol.wik.is/S3 ) to work on
<<quoted lines omitted: 7>>
> To unsubscribe from the list, just send an email to
> lists at rebol.com with unsubscribe as the subject.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[3/21] from: carl:cybercraft at: 20-Sep-2010 20:58
Hi Graham,
And thank you! I've not time to test Chris's code tonight, but it will be tested soon
and I'll let the list know how it behaves.
-- Carl Read.
On Monday, 20-September-2010 at 20:37:49 Graham Chiu wrote,
[4/21] from: compkarori::gmail at: 20-Sep-2010 20:57
Carl
the errors are not very informative, but the headers sent by Amazon do hold
the error information you need.
So, we need to modify the http protocol to give this to us .. eg.
create a global to capture the headers
protocol-http-header: none
at the top of the protocol, add this function
unless port/locals [port/locals: make object! [list: copy [] headers: none
querying: no]]
generic-proxy?: all [port/proxy/type = 'generic not none? port/proxy/host]
>>> add here
save-header: func [ obj ][
protocol-http-header: obj
obj
]
and change the success function to this:
success: [
headers: make string! 500
while [ ( line: pick port/sub-port 1 ) <> "" ] [append headers join line
^/
] ; remove the headers
port/locals/headers: headers: save-header Parse-Header HTTP-Header headers
port/size: 0
if port/locals/querying [if headers/Content-Length [port/size: load
headers/Content-Length]]
if error? try [port/date: parse-header-date headers/Last-Modified]
[port/date: none]
port/status: 'file
]
so, now when it fails, you can inspect the value of the protocol-http-header
to see what Amazon doesn't like about the authentication.
HTH
On Mon, Sep 20, 2010 at 8:37 PM, Graham Chiu <compkarori-gmail.com> wrote:
> Hi Carl
> I never tested on Linux ... but the forbidden error suggests that the
<<quoted lines omitted: 26>>
> http://www.compkarori.co.nz:8090/
> Synapse - the use from anywhere EMR.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[5/21] from: carl:cybercraft at: 20-Sep-2010 22:12
Will try that too Graham, hopefully tomorrow night, and will let you know the results.
-- Carl Read.
On Monday, 20-September-2010 at 20:57:24 Graham Chiu wrote,
[6/21] from: carl:cybercraft at: 21-Sep-2010 21:48
Hi again Graham,
No luck on all fronts, I'm afraid. Chris's protocol works on XP but not on Linux, same
as your program. And your header capture modification doesn't seem to be working, in
that protocol-http-header stays at none on Linux. (It gets the header object on XP.)
The error occurs in the Get-s3object function, it not being able to return result. (Which
you probably know.)
Also, success in the protocol doesn't seem to be called, suggesting the error happens
before it gets that far. I added a print comment to success before the line with save-header
in, and while it prints the comment in XP it doesn't in Linux.
Where to now? :)
-- Carl Read.
On Monday, 20-September-2010 at 20:57:24 Graham Chiu wrote,
[7/21] from: compkarori:gma:il at: 21-Sep-2010 21:59
Hi Carl
Well, if that is the case, then how about testing if the prot-http.r as
modified by me has a problem on Linux.
So, load the protocol and test to see if you can read any web pages and if
the protocol-http-header works then.
On Tue, Sep 21, 2010 at 9:48 PM, Carl Read <carl-cybercraft.co.nz> wrote:
> Hi again Graham,
> No luck on all fronts, I'm afraid. Chris's protocol works on XP but not on
<<quoted lines omitted: 87>>
> To unsubscribe from the list, just send an email to
> lists at rebol.com with unsubscribe as the subject.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[8/21] from: carl:cybercraft at: 21-Sep-2010 23:02
Hi Graham,
Yes, it'll load web pages ok on Linux. I started with a new instance of REBOL and read
a web page ok, then ran prot-http.r, then read another webpage without any problem. My
comment in success was printed too.
-- Carl Read.
On Tuesday, 21-September-2010 at 21:59:10 Graham Chiu wrote,
[9/21] from: compkarori:gma:il at: 21-Sep-2010 23:19
Hi Carl
I think you'll need to do a read/custom and see then if there
is a difference
On Tue, Sep 21, 2010 at 11:02 PM, Carl Read <carl-cybercraft.co.nz> wrote:
> Hi Graham,
> Yes, it'll load web pages ok on Linux. I started with a new instance of
<<quoted lines omitted: 11>>
> To unsubscribe from the list, just send an email to
> lists at rebol.com with unsubscribe as the subject.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[10/21] from: compkarori::gmail at: 22-Sep-2010 18:11
Hi Carl
I did a single test on Ubuntu and was able to download a file off S3 which
was protected.
So, perhaps the issue lies elsewhere?
On Tue, Sep 21, 2010 at 11:19 PM, Graham Chiu <compkarori-gmail.com> wrote:
> Hi Carl
> I think you'll need to do a read/custom and see then if there
<<quoted lines omitted: 29>>
> http://www.compkarori.co.nz:8090/
> Synapse - the use from anywhere EMR.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[11/21] from: carl:cybercraft at: 22-Sep-2010 21:29
Hi Graham,
Yes - elsewhere! I hadn't set the timezone on Linux, (it's a newish install), and that
was the problem. I noticed the time was four hours out in the authorization block, which
put me on the right track.
BTW, I've just remembered the url-encode function wasn't included with your script -
I had to find it elsewhere. It's not in View 2.7.7, but perhaps it's in other REBOLs?
Anyway, many thanks for your patience! And the script - useful.
-- Carl Read.
On Wednesday, 22-September-2010 at 18:11 Graham Chiu wrote,
[12/21] from: compkarori:gma:il at: 22-Sep-2010 21:50
Hi Carl
Yes, about that url-encode, I just added one now :)
The problem with the prot-http is that you do lose the headers returned
unless you open the url instead. Hence my need to modify the success
function.
The headers should have given the error information from Amazon that you
were using the incorrect time. So, not sure why that didn't work ..
On Wed, Sep 22, 2010 at 9:29 PM, Carl Read <carl-cybercraft.co.nz> wrote:
> Hi Graham,
> Yes - elsewhere! I hadn't set the timezone on Linux, (it's a newish
<<quoted lines omitted: 14>>
> To unsubscribe from the list, just send an email to
> lists at rebol.com with unsubscribe as the subject.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[13/21] from: carl:cybercraft at: 26-Sep-2010 13:10
Hi yet again Graham...
Today uploading to Amazon S3 stopped working for me on Windows XP! And after a while
it dawned on me why - daylight saving started here today. Sigh...
So, if I turn off daylight saving in Windows, (setting Windows' clock back an hour),
I can upload. But I can't if I then manually set the clock forward an hour. (And no matter
what settings I choose, REBOL's time always matches Windows' time.)
I noticed Windows says it's still only 12 hours ahead when daylight saving is ticked
as on (as does REBO), so I'm picking that's the source of the problem. REBOL grabs the
time-zone from Windows, when what it needs is 13 hours. Unless there's a setting for
the time-zone in REBOL? I couldn't one in the 'Viewtop'.
-- Carl Read.
On Wednesday, 22-September-2010 at 21:50:42 Graham Chiu wrote,
[14/21] from: carl:cybercraft at: 26-Sep-2010 13:34
Followup to this...
http://developer.amazonwebservices.com/connect/message.jspa?messageID=119866
Seems the best approach would be to get the time from the Amazon server, thus bypassing
any local time issues.
-- Carl Read.
PS. And yes, I meant REBOL below, not REBO, and the last sentence should've read "I couldn't
/find/ one in the 'Viewtop'."... (Well what'ya expect on the day daylight saving cuts
in!)
On Sunday, 26-September-2010 at 13:10:55 Carl Read wrote,
[15/21] from: compkarori:g:mail at: 26-Sep-2010 12:55
Carl
If it is any consolation to you, R3 shows the timezone as +13 even as R2
shows it as +12 :(
I always set the time to NIST time in my applications before I do anything
else.
On Sun, Sep 26, 2010 at 1:34 PM, Carl Read <carl-cybercraft.co.nz> wrote:
> Followup to this...
> http://developer.amazonwebservices.com/connect/message.jspa?messageID=119866
<<quoted lines omitted: 73>>
> To unsubscribe from the list, just send an email to
> lists at rebol.com with unsubscribe as the subject.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[16/21] from: compkarori:gmai:l at: 26-Sep-2010 13:31
Hi Carl
Try this
In the S3 tools, change now-gmt to
now-gmt: func [ t ][ t: t - t/zone t/zone: 0:00 t ]
run ladislav's nist tools
do http://www.fm.tul.cz/~ladislav/rebol/nistclock.r
and then change all occurrences of now-gmt to now-gmt nist-corrected-time
On Sun, Sep 26, 2010 at 12:55 PM, Graham Chiu <compkarori-gmail.com> wrote:
> Carl
> If it is any consolation to you, R3 shows the timezone as +13 even as R2
<<quoted lines omitted: 95>>
> http://www.compkarori.co.nz:8090/
> Synapse - the use from anywhere EMR.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[17/21] from: carl:cybercraft at: 26-Sep-2010 8:13
Hi Graham,
It's fixed itself, (at least for now:), in that uploading's working with no change to
your script and Windows set to daylight saving. Maybe something was wrong at Amazon's
end, or they received a swag of complaints from NZ and just broadened their time compliance
until it's sorted out.
And I've just read your new mail with ladislav's nist clock fix. I'll test that, (though
maybe not today.), and let you know how it behaves.
Thanks again,
-- Carl Read.
On Sunday, 26-September-2010 at 12:55:04 Graham Chiu wrote,
[18/21] from: carl:cybercraft at: 2-Oct-2010 22:03
Hi Graham,
The script in the state it was before your following changes worked ok all this week,
(except for the few hours after the daylight-saving-time change last Sunday as mentioned),
and it also works with your following changes. I'll be sticking with the earlier version
though, as I find the wait with connecting to the nist clock very annoying. REBOL scripts
should not take many seconds to get going! :)
Many thanks,
-- Carl Read.
On Sunday, 26-September-2010 at 13:31:57 Graham Chiu wrote,
[19/21] from: compkarori::gmail at: 2-Oct-2010 22:03
Hi Carl
It was very odd ... Rebol was reporting TZ as +12 for part of the day, and
then it suddenly changed to +13:00!
I was watching it on Altme too .. posts were out by an hour and then
suddenly they came into sync.
Check your tz now, I'm guessing it is now +13:00
Probably an obscure Rebol bug.
On Sun, Oct 3, 2010 at 12:18 PM, Carl Read <carl-cybercraft.co.nz> wrote:
> Hi Graham,
> The script in the state it was before your following changes worked ok all
<<quoted lines omitted: 34>>
> To unsubscribe from the list, just send an email to
> lists at rebol.com with unsubscribe as the subject.
--
Graham Chiu
http://www.compkarori.co.nz:8090/
Synapse - the use from anywhere EMR.
[20/21] from: carl:cybercraft at: 2-Oct-2010 22:03
On Sunday, 3-October-2010 at 13:12:09 Graham Chiu wrote,
>Hi Carl
>
>It was very odd ... Rebol was reporting TZ as +12 for part of the day, and
>then it suddenly changed to +13:00!
>I was watching it on Altme too .. posts were out by an hour and then
>suddenly they came into sync.
>
>Check your tz now, I'm guessing it is now +13:00
Hmm. Yes, it is +13:00 now!
>Probably an obscure Rebol bug.
More likely a Windows' one, since REBOL goes through Windows to get the time, I'm assuming.
And when daylight saving ended this year in April, this XP's clock was two hours out!
And according to Microsoft's webpage, clocks are put forward in October in the Southern
Hemisphere, despite ours moving forward in September...
http://support.microsoft.com/gp/cp_dst
But then, daylight saving's voodoo, anyway. You're saying there's a day a year with an
hour missing from it? Oh yes, and not only that, but there's an hour once a year that
repeats itself. "Yeah, right!"
-- Carl Read.
[21/21] from: moliad::gmail at: 4-Oct-2010 16:43
windows DST Handling is idiotic... it actually moves the hardware
clock instead of just correcting itself according to TZ information.
its that stupid.
This has always been a problem and indeed, because of this some clock
times really do re-occur twice which mixes up many applications.
At one place where I worked, the db admin would shutdown the server
during DST switch night and bring it back up the next morning...
otherwise it would take him all day to fix all the db time breakups
and license servers which all failed starting at midnight :-)
-MAx
On Sat, Oct 2, 2010 at 8:35 PM, Carl Read <carl-cybercraft.co.nz> wrote:
> On Sunday, 3-October-2010 at 13:12:09 Graham Chiu wrote,
>>Hi Carl
<<quoted lines omitted: 10>>
> http://support.microsoft.com/gp/cp_dst
> But then, daylight saving's voodoo, anyway. You're saying there's a day a year with
an hour missing from it? Oh yes, and not only that, but there's an hour once a year that
repeats itself. "Yeah, right!"
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted