AltME groups: search

Has anyone written a util that spiders a ftp site and prints out 
the space taken within each directory?

even with a ftp client?

some ftp clients NEVER show files starting with a dot, others need 
to be configured to show them.

Bo has written such a utlity .. that also backs up to a ftp directory

I'm looking for a good FTP Client for XP.
Any suggestions?

TotalCommander is not bad, but misses secure FTP connection. FileZilla 
or UltraFXP were recommended to me.

What about Reichart's FTP client?

If I FTP then by default the change.

damn my ftp server is sick ....

wasn't there an FTP group here before?

some code supplied from anton that I was about to use.... has disapeared... 
I'm pretty sure in was in a group called ftp

Is there a bug in the ftp protocol?  I tried opening a different 
port viz 4559, but it still opens up on port 21.

Well, there is _at least_ one bug in the FTP protocol.  I haven't 
run into the one you are talking about, but I think I did solve the 
one that I was having a problem with (425 response from the server 
causing Rebol to wait infinitely).

opening a ftp port using the scheme: 'ftp and port: 4559 still opens 
up port 21 according to trace/net

Yes, I've installed Hylafax which uses a modifed ftp protocol on 
port 4559.

That's interesting .. it works that way, but not using [ scheme: 
'ftp port: 4559 host: 192.168.1.252 user: "user" pass: "password 
]

>> read ftp://Graham:[password-:-192-:-168-:-1-:-252]:4559
URL Parse: Graham password 192.168.1.252 4559 none none
Net-log: ["Opening" "tcp" "for" "FTP"]
Net-log: [none ["220" "230"]]

Net-log: {220 localhost.localdomain server (HylaFAX (tm) Version 
4.2.1) ready.}

which is working, but trying this ..


>> port: open [ scheme: 'ftp port: 4559 host: 192.168.1.252 user: 
"Graham" pass: "password" ]
Net-log: ["Opening" "tcp" "for" "ftp"]
Net-log: [none ["220" "230"]]
Net-log: "220 Microsoft FTP Service"
Net-log: [["USER" port/user] "331"]
Net-log: "331 Password required for Graham."
Net-log: [["PASS" port/pass] "230"]

doesn't because the MS ftp server at port 21 is answering instead

What is the correct code for ftp to a site when the user id is [john-:-smith]?

Something like:
read [scheme: 'ftp user: "[john-:-smith]" pass: "****" host: "_____"]

Because I'm writing scripts on more than one computer I need to sync 
files somehow. I can use flashdisk for synchronization, but USB is 
not always available or I forget my flashdisk at home, so it's not 
always the right option.

Or I can use ftp to upload and download files. But at the end I've 
got lots of different directories with different versions, because 
I have no intelligent file structure.

I was inspired by Google filesystems for win and lin so I decided 
to use some freemail (gmail preferably) for my scripts maintaing. 
Unfortunatly, Gmail needs some authentication, SSL or what and SSL 
under Rebol needs Command and Command needs 350$ to buy.

So I found another freemail provider that offers both non-authenticated 
SMPT and POP and therefore is OK for REBOL (btw. remeber the old 
REBOL example? send [luke-:-rebol-:-com] read http://www.rebol.com? Hard 
to do with all the authetications required today.) and I started 
coding.

The result is a small application called %rspace.r that can upload 
file to repository, download newest version from repository, or you 
can get list of all files in repository and finally, if you're happy 
with your script, you can publish it on www/ftp. All this with documentation 
in less than 6kB.

All you need is REBOL and mail account cappable of SMTP/POP without 
authentication. It's good to have an FTP account for publishing files 
but that's not required. If you do not have an mail account, I've 
set up one on seznam.cz, user 'rebolspace' and pass 'spacerebol' 
for testing this application (it's built in, so you can start testing 
right after download).

Remember, it's just alpha, does not have many features, but it works, 
I can write something here, update it there and have all the versions 
accesible from everywhere. It's written for REBOL scripts so with 
big projects it's going to be very slow and unusable, but for small 
project (and most REBOL scripts are really small) it's probably good.

So download it form http://krutek.info/rebol/rspace.r(stable) or 
http://rebolspace.sweb.cz/rspace.r(latest published version). 

WARNING: because [rebolspace-:-seznam-:-cz] is open account it won't be 
wise to use it ordinarily. Please, if you like it, set up your own 
account and use it instead of built-in one.
And remember: all suggestions and fixes are welcome.

I am trying right now to write a file to an FTP server. What I would 
like to do is:
- open the port 
- try to write the file
- if that fails, create the parent directory if necessary
- try to write the file again
- close the port

... and I have my head deep in FTP handler... :)

does make-dir/deep not work on FTP sites?

No, and it caused me grief. But there is help at hand !  I patched 
make-dir to support FTP.

i think there are issues with ftp as a url path is not guaranteed 
to match the file system path

A URL path is _supposed_ to map to the file system path _if_ the 
scheme is ftp://
/
 has no assumed hierachical meaning in other URL schemes.
Hence the confusion at times.

I think REBOL is playing safe in not assuming the mapping. And, given 
how badly many FTP clients are written, that is probably for the 
best.

is it possible to change file permissions via FTP with rebol?

We need to yell at Dan, and get him to post the source to FTPGadget. 
 It has examples of just about everything for FTP.

I have just managed to patch FTP handler so it creates subdirectories 
recursively as needed. So code like this, which would fail before 
if the directory didn't exist, now works:
write ftp://user:[pass-:-server-:-dom]/my-dir/test "hello"

Is there any script for ftp upload of large files?

Does that work for FTP ?

the trick is, thet there must be the /new switch in the ftp port 
(as at least my ftp server do not support append)

Hm, but now how to make directory thru FTP:(

Just be aware that make-dir/deep usually doesn't work with FTP

I found that make-dir/deep doesn't work, so I patched FTP OPEN so 
that it creates deep paths as necessary.

http://home.wilddsl.net.au/anton/rebol/patch/ftp-open-new-patch.r

port: make port! ftp-spec

port/state/flags: port/state/flags or system/standard/port-flags/open-new
port/handler/open port

Anton, yout ftp patch was not working, but I solved the issue with 
missing directories using this code:
while [

 error? set/any 'err try [trgp: open/direct/new/write rejoin [ftp-url 
 trg-dir trg-file]]
][
	err: disarm err
	if all [
		err/code = 800
		parse err/arg1 [thru "tcp 550 " copy missingdir to ":" to end]
	][
		print ["Making directory:" join ftp-url missingdir]
		if error? try [make-dir join ftp-url missingdir][
			trgp: none
			break
		]
	]
]

Oldes, what FTP server did you try it with ?

How badly was my ftp patch not working ? :-)

Are you sure the patch was applied correctly ? (uncomment the   ;print 
"patch"   line.)  Perhaps your FTP scheme is already patched ?

Sorry, that wasn't very clear..  Use my code snippet above just to 
create the path.
	port: make port! ftp-spec  ; only path, no target

 port/state/flags: port/state/flags or system/standard/port-flags/open-new
	port/handler/open port
	close port
; now try to open/direct ... etc..

I found the problem, you have this:

{Server error: tcp 550 Can't change directory to} thru {:} {No such 
file or directory}
but my ftp server's response is only:

Server error: tcp 550 httpdocs/test/1/2/: No such file or directory

Aha..! yes, very good. I knew that piece of code was possibly brittle. 
(Thankyou FTP, thankyou.)

What's your FTP server, by the way ?

(I mean, what's the name and version of the FTP server ?)

I could if I didn't need to process the error contents. For example, 
I try to connect to an FTP server using an URL with user/pass in 
it. This is normally hidden from view, but if the URL fails, the 
entire user:[pass-:-url] is passed to the error object in clear text.

I'm trying to log into an FTP server with \ in the user name. Is 
that legal to use? I've tried a few different clients and only Total 
Commander will accept it. Rebol will not accept it as a valid URL, 
unless the \ is removed.


The problem is that the webhost Talkactive apparently use \ in all 
their usernames...

if \ breaks something in the URL parser, then it would be a problem 
of course, but then again, you can't rely that much on the FTP system 
in rebol

That's a problem with FTP in general. There are some servers which 
break the standard (which is also open to interpretation in some 
areas). RT's url parser is doing the correct thing, but supporting 
FTP in the real-world means also dealing with "rogue" standard-breaking 
servers. You could argue that if RT includes FTP in the language 
they should go the whole way with it to prevent dashed expectations. 
On the other hand, you can see rebol is more about breaking with 
the past and coming up with new, more modern (and hopefully more 
reliable) protocols. (Of course, it is possible to have both.)

I would suggest an FTP powerpack then with as many bells and whistles 
as possible.

I would check out FTP-Gadget, which was open-sourced by Reichart. 
(Where did I get it..? Qtask .. ?)

So, I expect there to be a number of servers supported by FTP-Gadget.

henrik, use a block, not a url, i.e. open [scheme: 'ftp user: "\\\" 
...]

I am just transferring many files via ftp to a FTPZilla on Windows.
Aftert some time the ftp hangs.
If I use one steady connection, it hangs writing.

If I use one connection per file, it hangs with the message connecting.
Anyone seen a similar behaviour ?

I have an interesting problem with FTP.  I'm using FileZilla on my 
server and I get the following entry in the log file:

Rebol, never returns from that.  It seems to be a bug in Rebol's 
FTP handler as it doesn't ever time out and continue, it just locks 
up.  I have to press "Escape" to get out of it.  I've left it for 
up to 12 hours and it still doesn't return.

are you using the latest version 1.3.2 of rebol? I think there are 
some serious patches to FTP somewhere, but I can't remember if they 
made it official.

I'm using the SDK and I'm including sdk-2-6-2/source/prot.r which 
should have the FTP protocol in it.  Does anyone know if there is 
a newer version of prot.r other than sdk-2-6-2?

I think all of Romano's patches http://www.rebol.it/~romano/ftp-patch.r
were incorporated already.

Well, I haven't tried it with a different FTP server, but it definitely 
appears the problem is on the Rebol side.

I'm not even sure it is an FTP issue.  The timeout should be triggered 
by Rebol's networking kernel, shouldn't it?

(I meant "I'm not even sure it is a Rebol FTP protocol issue")

Also, I am using FTP in passive mode.

I am overwhelmed with projects presently, and I'm willing to pay 
if someone can come up with a patch to prot.r for FTP (if that is 
where the problem exists).  I would also allow the patched prot.r 
to be released back to the community.

the problem is that it may be just with a particular ftp server ...

But it seems the FTP server is working properly according to the 
log file.  If it cannot retrieve the directory listing, it sends 
a 425 and then eventually disconnects.  In any case, Rebol should 
never hang indefinitely even if the FTP server misbehaves.

I'll see if I can upload it.

 once when i wanted to upload i had to search ftp and password etc 
 and that took time.

the reason for the // is to allow relative paths like: //www.rebol.com/ 
  where the scheme is the same as the base url. Nobody has ever used 
this; also, it could have been achieved by using :www.rebol.com/ 
instead... so, yeah, it was not really a good idea. I also don't 
think ftp:file.txt (meaning, change scheme, but keep host and path) 
has ever been used and not sure it's supported by software. so in 
practice http:www.rebol.com/ would have worked.

The third release of Syllable Server has been published. This is 
an important release, because it is the first one that focused on 
making the system actually usable as a server. A number of popular 
servers were added and configured, and also several innovative REBOL 
software stacks. Out of the imaginary box, Syllable Server is now 
ready for such things as accepting remote SSH log-ins over the network, 
running a web server on the Cheyenne REBOL server, running an FTP 
server and several more. Special attention has been paid to programmability, 
with support for developing Model-View-Controller web applications 
in QuarterMaster and networking applications with the REBOL/Services 
Service Oriented Architecture. The Genode Nitpicker windowing system 
is also included. Read the rest in the full changelog. An extensive 
manual was also written, which is easy to follow. As usual, both 
a BitTorrent download (preferred) and a regular download are available 
(80 MB 7-Zip archive). Please use the torrent if you can.

This release focuses on making the system usable for running a number 
of standard servers, and several innovative REBOL servers.


The development files of the system, program headers, static libraries 
and development documentation, were moved to a separate area in /system/development/ 
and are now shipped in a separate package. If you want to compile 
software on Syllable Server, you need to install and register this 
package. The development files need to match the system: you can't 
use a package of any other Syllable version. (You will also need 
to install the Developer's Delight package collection and possibly 
other packages.)


User directories were moved from /home/ to /users/. Resource packages 
are in the process of moving from /usr/ to /resources/. /resources/ 
is currently a symbolic link to /usr/ so that resource packages will 
work from both places during the migration.


Many fixes were made, including more fixes for the CUPS print server 
and GhostScript. Creation of extra user accounts is possible now.


Many packages were updated, including the Linux kernel, IPTables, 
the GCC libraries, OpenSSH, SDL and QEmu. DirectFB was not upgraded 
due to incompatibility with Links2.


CDRTools were included for burning CDs, and the NetCat networking 
tool and the Transmission BitTorrent client were added.


In addition to the Syllable-specific early initialisation scripts 
(in the early-init subdirectory of packages), the late initialisation 
scripts (in the init subdirectory of packages) are now also executed. 
Several more initialisation scripts from Linux From Scratch were 
also added. Some servers can be started with the LFS scripts, others 
with the Syllable scripts (this will be unified in later releases).


The OpenSSH server was configured and now runs by default. At the 
first system start, security keys are generated that identify the 
server.


A collection of well-known root certificates from Certification Authorities 
was added to allow OpenSSL-based programs (such as OpenSSH) to establish 
the identity of destination points for network connections.


A MIME-types database was added in /etc/mime.types that is used by 
many programs, such as web servers, to identify the MIME types of 
files based on their file name extensions.

Several REBOL software stacks were added:

- The REBOL/Services Service Oriented Architecture.

- The UniServe network server framework.
- The Cheyenne Apache-class web server.
- A CAPTCHA library.
- A MySQL network protocol.

- The QuarterMaster web programming framework, based on a Model-View-Controller 
architecture. By default, it's configured to run on Cheyenne.


- The TINY library for parsing text, abstracting data access and 
building templates of generic text formats (including HTML). This 
library is an original creation and targets both ORCA and REBOL.


Configurations, including initialisation scripts, were added for 
the OpenSSH remote access server, the CUPS print server, the BIND 
domain name server, the Apache web server, the RSync file synchronisation 
server, the SaMBa Windows-compatible file server, the INetUtils FTP 
server and the VSFTP FTP server. Several of these are not included 
in the system, but need to be installed separately (the system is 
prepared for them). The sshd, cupsd and initd servers are started 
by default.


S3Cmd/S3Sync was included, a tool for accessing the Amazon Simple 
Storage Service (S3) and synchronising files with it.


As a demo, the Genode operating system framework, its Nitpicker windowing 
server (built on SDL) and its demonstration programs were included.

It's running Cheyenne, possibly with QuarterMaster, FTP and an SSH 
server

Dunno, at least this one is not business critical. A decade ago, 
Syllable's forerunner was already running as a public web/FTP/CVS 
server

If you could settle for an FTP server you could try Desktop, but 
you can't trust your files to be safe

Yes. On our personal machines, we're doing development and office-type 
work. On our servers, we're running AltME and Cheyenne with supporting 
servers such as OpenSSH and FTP. We also have BitTorrents running 
to support Syllable downloads

Kaj, in your documentation, you mentioned how to start Cheyenne and 
the FTP server automatically by uncommenting lines in their respective 
start scripts. On startup, where are these scripts being called? 
I ask because I would like to add applications to run at startup. 
If it's a standard Linux thing, that's fine, I can look it up. I 
noticed that the directory structure, though, is different from Linux 
installations I've seen in the past.

Eg. for ftp-protocol, change this 


 open-check: [none ["220" "230"] ["USER" port/user] "331" ["PASS" 
 port/pass] "230" "SYST" "*"]

to


 open-check: [none ["220" "230"] ["USER" dehex port/user] "331" ["PASS" 
 port/pass] "230" "SYST" "*"]

well, it works for me... doing a trace/net .. I saw the username 
being passed correctly to the ftp server

>> read ftp://gchiu%2540compkarori.co.nz:[password-:-ftp-:-rebol-:-com]

URL Parse: gchiu%40compkarori.co.nz password ftp.rebol.com none none 
none
Net-log: ["Opening" "tcp" "for" "FTP"]
connecting to: ftp.rebol.com
Net-log: [none ["220" "230"]]

Net-log: {220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------}
Net-log: "220-You are user number 3 of 150 allowed."
Net-log: "220-Local time is now 03:37. Server port: 21."
Net-log: "220-This is a private system - No anonymous login"
Net-log: {220-IPv6 connections are also welcome on this server.}

Net-log: {220 You will be disconnected after 30 minutes of inactivity.}
Net-log: [["USER" dehex port/user] "331"]
Net-log: {331 User [gchiu-:-compkarori-:-co-:-nz] OK. Password required}
Net-log: [["PASS" port/pass] "230"]
** User Error: Server error: tcp 530 Login authentication failed

** Near: read ftp://gchiu%40compkarori.co.nz:[password-:-ftp-:-rebol-:-com]

I thought one or more of you gurus would have some advice. I'm trying 
to back up a website via ftp to move all the files to another server. 
Some of the files have owner "apache" and won't let me move them. 
There are a lot of those files in various directories. Not being 
a linux guy, is there an easy way to chown a whole directory with 
subdirectories of files?

chown -R src dest 


and if you can use scp or rcp (depending on your setup) to copy your 
files over, it should be less hassle than ftp which often has many 
issues with stranger filenames and permissions.


on one server, for example, a file was created with ftp, which is 
a legal (albeit twisted) path, but it cannot be access or deleted 
from that same ftp access, because the path gets mangled when it 
goes through the url path parser of that ftp server.


also unless both source and destination ftp servers are the same, 
you can have other nasties.


its always best (and much faster) to gzip your whole directory and 
copy over one file, and then unpack it on the other server (you also 
get a free backup ;-)

putty comes with an scp file xfer utility. I'd recommend using that 
instead of ftp... its really easy to use... just look in the docs 
everything is there.


this also has the advantage that you're not sending your password 
in clear text over the web (which ftp gladly does for you :-)

guys, can you see how you made your life just more complicated? :-) 
We could use simple ftp or so :-)

that chatting was about how to patch, not how to get it to me.

when you have a list of typical commands, eg ctx-menu checkout, its 
as easy as ftp

the rest is guru-stuff. cu, trying to find the diff in my ftp-client 
;)

More easy. Less ftp-compatibility issues.

i prefer torouise-integration about looking for an ftp-client. even 
if that one is inbuild n explorer.

Having said that, I think for my R3 ftp implementation, I stream 
the file off the file system instead of reading into a variable first

Gabriele - no, no bug fixes only - R3 alpha has one and only scheme 
- http - where is pop, ftp, smtp?

petr, what would having pop or imap change? (btw, ftp... that's a 
mess of a protocol. find someone wanting to write that one :P) i 
really don't understand the point. pop and imap are "trivial". VID 
is the focus. i can't wast time on pop now. that can be done later 
on. i must spend my time on what's important - VID. having an R2-like 
R3 is just crazy. R2 is already here.

You act like anyone is able to write imap or pop protocols themselves. 
And yes, ftp might be the mess, so we are not going to have one, 
right?

Pekr, you can say that about many things. There are many protocols. 
Which one should go in first? HTTP was logical here. Now which one 
goes next? If Gabriele spends time on FTP and not on VID, I can't 
work on skinning. It could be other things, but VID is far more important 
right now than FTP, because FTP is probably fairly trivial to do, 
but still a one-man project that should be done when Gabriele can 
be free to do that.

If someone needs pop, imap or ftp now, probably he will write it 
self. If it's not good enough for official R3, discard it. Someone 
else will try again.

Ingo - I don't agree at all. It is like saying everyone CAN code 
in C, because you have plenty of free text editors available. You 
would not probably want to see My rebol code and surely you would 
not accept it being part of official distro. So really - No - not 
everybody CAN code pop, imap protocol. If so - why did it take so 
long to get proper FTP protocol ready? Apop? etc.?

yes, ftp is so trivial, that we can't get it right for 10 years :-)

Henrik - as for FTP - I think that Gabriele and Romano and Reichart 
might be actually right, that FTP is a real mess :-)

yes, it is unfortunately a mess. I'm not sure what should be done 
with it, other than be implemented by someone who is an expert on 
the FTP protocol, rather than just implement a rudimentary one that 
follows the RFC.

Ehm, I actually worked on the FTP scheme, testing it for Romano who 
did a LOT of bugfixing on it

In fact it's probably the answer to the FTP discussion we had earlier.

I gather one day it might be possible to write a non-standard ftp 
hylafax protocol ... but in the meantime I guess I'm restricted to 
running the commandline utilities.