File writing issue
[1/15] from: mattsmac::hotmail::com at: 29-Oct-2003 16:36
Ok here's one more for everyone to ponder. When you write a file in rebol
that is somewhat large (around 1 meg or so) and then immediately after
writing it you want to do something with it (email it, etc) I get an error
message that the file is not there. On inspection of the directory where it
is supposed to be, it is there. So I'm guessing there is a little delay
between the time when you write the file (and rebol returns) until the file
is really accessable. So I put this line of code in to counteract that
while [not exists? file] []
to make it wait for the file to be there.
So now I can do things with the file, but not the whole file. When I email
it it usually shows up as 64B. So then I entered this line of code
while [size? file <> fsize] [] ; where fsize is how big the file should be,
thinking that the file might get written to the disk in peices so I want to
make sure the whole file is there before doing stuff with it. This didn't
work either, it did wait, but the file still doesn't come through complete.
Any suggestions?
Matt
[2/15] from: andrew:martin:colenso:school at: 24-Dec-2003 22:43
Matt wrote:
> Ok here's one more for everyone to ponder. When you write a file in
rebol that is somewhat large (around 1 meg or so) and then immediately
after writing it you want to do something with it (email it, etc) I get
an error message that the file is not there.
> ...
> Any suggestions?
I get the distinct impression that there's something wrong with your
computer's operating system! I've had no trouble writing extremely large
files with Rebol and immediately using the file. The only "trouble" that
I've had in this area is writing a large number of large files all at
once, when the operating system takes a while to show all the icons for
the files! But that's not the fault of Rebol.
I'd suggest that you look very closely at your computer virus checker,
as it could delaying the write of files while it scans for virii. I had
this problem with a virus scanner which persistently wanted to scan all
files that were being written; eventually I had to remove my virus
scanner because it grew incapable of handling the load.
Andrew J Martin
Attendance Officer
Speaking in tongues and performing miracles.
Colenso High School
Arnold Street, Napier.
Tel: 64-6-8310180 ext 826
Fax: 64-6-8336759
http://colenso.net/scripts/Wiki.r?AJM
http://www.colenso.school.nz/
DISCLAIMER: Colenso High School and its Board of Trustees is not responsible (or legally
liable) for materials distributed to or acquired from user e-mail accounts. You can report
any
misuse of an e-mail account to our ICT Manager and the complaint will be investigated.
(Misuse can come in many forms, but can be viewed as any material sent/received that
indicate or suggest pornography, unethical or illegal solicitation, racism, sexism, inappropriate
language and/or other issues described in our Acceptable Use Policy.)
All outgoing messages are certified virus-free by McAfee GroupShield Exchange 5.10.285.0
Phone: +64 6 843 5095 or Fax: +64 6 833 6759 or E-mail: [postmaster--colenso--school--nz]
[3/15] from: rebol:techscribe at: 29-Oct-2003 14:04
Hi Matt.
If you copy the file manually to the directory, and then attempt to
email it using REBOL, do you get correct results?
If not, how are you determining how many bytes you are sending out?
Does the fiile contain binary data or ascii?
Are you emailing it as an attachment, or as the text body?
Elan
Matt MacDonald wrote:
[4/15] from: mattsmac:hotma:il at: 30-Oct-2003 8:06
If the file is there allready it emails fine. Its just when I do a
read/binary/direct from one location and a write/binary/direct from another
location and then immediately try to do anything with it that it gives me
issues.
Matt
--------------------------------
Hi Matt.
If you copy the file manually to the directory, and then attempt to
email it using REBOL, do you get correct results?
If not, how are you determining how many bytes you are sending out?
Does the fiile contain binary data or ascii?
Are you emailing it as an attachment, or as the text body?
Elan
Matt MacDonald wrote:
[5/15] from: mattsmac:hotm:ail at: 30-Oct-2003 8:31
I am using the "build-attach-body" function if that makes a difference. I
need to put more into the headers of the email than the standard send
function will let me do, like a delivery confirmation header. If I remember
correctly though, I had the same problem when using the standard send
function. The files are mixed types, some are ascii, some are binary, but
that should be taken care of by the function right?
Matt
[6/15] from: mattsmac:ho:tmail at: 30-Oct-2003 15:00
So any other toughts on this??
[7/15] from: antonr:iinet:au at: 31-Oct-2003 12:55
I suspect the problem is either in your
/direct mode handling, or the build attach
code.
What happens when you do not use /direct mode?
If you write in buffered mode, you have the file
contents in memory, so you should be able to
write/binary the file, then send straight from memory.
Also you can read/binary and compare the contents
in memory.
contents: read/binary file
write/binary file2 contents
contents2: read/binary file2
; compare contents
print (checksum contents) = checksum contents2
I know you want to use /direct mode, but until you
are bug free... divide and conquer.
Anton.
[8/15] from: mattsmac:hotma:il at: 31-Oct-2003 8:57
That makes sense, but how do I email an attachment that is in memory.
Basically this is the scenario:
The client software sends a file to the server, this needs to be done using
read and write commands because FTP is not secure enough. Then once the
file gets to the server, the server sends out an email which includes the
file. So as far as I can see, the file needs to come from disk and not from
memory. I have tried doing a read/binary of the file into a string in
memory and manually formatting the message body, but it never comes out as
the attachment in the email, its just always a text file that has all the
binary in it.
Matt
------------------------------------
I suspect the problem is either in your
/direct mode handling, or the build attach
code.
What happens when you do not use /direct mode?
If you write in buffered mode, you have the file
contents in memory, so you should be able to
write/binary the file, then send straight from memory.
Also you can read/binary and compare the contents
in memory.
contents: read/binary file
write/binary file2 contents
contents2: read/binary file2
; compare contents
print (checksum contents) = checksum contents2
I know you want to use /direct mode, but until you
are bug free... divide and conquer.
Anton.
[9/15] from: mattsmac:hotma:il at: 31-Oct-2003 9:47
Ok, this kindof works, if I put a wait 10 after I write the file to disk,
but before I email it, the email comes through fine (granted it takes 10
seconds longer, but I'm willing to deal with that for now). The problem
here is that even though everything works ok, I get a network timeout error.
How do I increase the network timeout window? Or is this a really bad
approach.
Matt
[10/15] from: antonr:iinet:au at: 2-Nov-2003 1:19
Right, you're not being terribly clear
what you are trying to do. Please explain
the system you want to implement.
What kind of machines are you using and have control over?
Do you control the server?
Let's also see the code.
This sounds like it should be really easy.
By the way, read and write is used in all the
network protocols, including FTP.
Anton.
[11/15] from: brett:codeconscious at: 2-Nov-2003 12:54
Hi Matt,
As Anton wrote - show us some code - preferrably the minimum amount of code
that demonstrates the problem. Doing so avoids a lot of ambiguity.
Anyway, I've just gone through all your messages.
You wrote that you actually had it working at one point but the whole file
was not there, and that you "send" the file from the client to the server.
So I guess you mean that your client writes to a server's shared hard disk
via Netbeui for example. If so, for testing, replace the client's writing
process with a batch file to copy a test file from the client to the server
over the network - in this way you isolate the REBOL client program. You can
do the same thing at the server end, instead of sending an email, copy the
recevied
file to another directory. In this way you isolate your problem
into REBOL behaviour or Operating system behaviour.
There was a problem I encountered at work some years back when we installed
NT4 servers. This was this little behaviour where the server would delay the
writing of files to disks in order to increase performance. Problem was that
in a shared files situation some programs esp. MS Word had their file
locking defeated. There was a registry fix for the server. Now that
behaviour was before Service Pack 1 even (I think) and I don't know if it is
relevent in later versions. So it might be completely irrevelent to you -
especially if your server is not Windows NT4 - but we wait with held breath
to discover what your server actually is :^)
Regards,
Brett
[12/15] from: mattsmac:hot:mail at: 3-Nov-2003 10:04
Sorry for the lack of clarity. Basically, I am implementing a secure
messaging system. This system has 3 components: client software, a
messaging server, and a file server. The messaging server handles most of
the database access of storing and retrieving the messages. The file server
is only for sending and recieving files that users want to send with the
messages. These files are stored on a local drive on the server computer.
This system will be used inside and outside of our company's network. When
used internally, rather than making the users use the client software, I am
sending emails to our exchange server to deliver the messages via Outlook.
So basically what is happening is that once the message is submitted by the
user (external) the message needs to be immediately emailed to the intended
internal user. A temporary solution I'm using is to give internal users
shared access to the servers file store and just insert links to that
location in their emails rather than actually attaching the file. I control
the server, and all machines are windows based machines.
Here is some code
Client:
insert port encrypt read/binary/direct fpath
insert port "done"
fpath is the client path to the file on the client computer
File server:
make-dir to-file dir
file: rejoin [dir fname]
replace/all file " " "_"
write/binary/direct to-file file decrypt read/binary/direct %temp.dat
where %temp.dat is an encrypted version of the file - the decrypt function
works fine, the file stores fine. dir is the location on the server's drive
to store the file.
messaging server:
;compose the MIME formatted message
m: ""
append m "EHLO^/"
append m rejoin ["MAIL FROM:<" return_email ">^/"]
append m rejoin ["RCPT TO:<" user_email ">^/"]
append m "DATA^/"
append m "MIME-Version: 1.0^/"
append m "Content-Type: multipart/mixed; boundary=unique-boundary-1^/"
append m rejoin ["Subject: " subj "^/"]
append m "X-MS-Has-Attach: yes^/"
append m rejoin ["Date: " tstamp "^/"]
append m rejoin ["From: <" return_email "^/"]
append m rejoin ["To: <" user_email ">^/"]
append m build-attach-body message files "--unique-boundary-1"
smtp: open/lines tcp://10.10.1.8:25
insert smtp m
insert smtp "."
insert smtp "QUIT"
close smtp
it is at this point that i normally get the error that the specified file
cannot be opened, but when I go to the location on the drive, it is there
and I can open it manually to verify the contents. If this problem can't be
solved it's not a huge problem because for most purposes, the work-around I
have going now is sufficient. It just seems like this should not happen.
Thanks to you both,
Matt
-------------------------------------------------------
Right, you're not being terribly clear
what you are trying to do. Please explain
the system you want to implement.
What kind of machines are you using and have control over?
Do you control the server?
Let's also see the code.
This sounds like it should be really easy.
By the way, read and write is used in all the
network protocols, including FTP.
Anton.
[13/15] from: brett:codeconscious at: 5-Nov-2003 11:32
Hi Matt,
A few notes on your email.
1) Your HELO line is misspelt below to be EHLO.
2) Your code just blindly "dumps" the HELO along with MAIL FROM and RCPT TO
and the rest of the message down the line - it does not wait on and check
for server responses to these first three. With SMTP, HELO and QUIT could
take a little while to occur. On the HELO the SMTP server needs to acquire a
lock on the mailbox, on the QUIT it processes any deletions and releases the
lock (who knows what else it could be doing). So with multiple clients
coming in, you really need to wait to check that you get the lock.
3) Why not use SEND instead of your own code? It will wait on the HELO and
check the response. You can add custom headers in using the /HEADER
refinement of SEND.
4) Have you checked that the process your server code runs in has the
necessary privileges? Perhaps you should add some code to write a log to a
text file - to ensure that that it is doing what you hope it is doing.
Regards,
Brett.
[14/15] from: antonr:iinet:au at: 5-Nov-2003 17:43
Just a few notes on the multiple appends for now;
> messaging server:
> ;compose the MIME formatted message
<<quoted lines omitted: 16>>
> insert smtp "QUIT"
> close smtp
You can join them all together with rejoin:
m: copy "" ; copy ensures it a fresh new empty string
append m rejoin [
"HELO^/"
"MAIL FROM:<" return_email ">^/"
"RCPT TO:<" user_email ">^/"
"DATA^/"
; ... etc.
]
Also this could be rewritten to add your newlines for you
won't save you much code in this case, though:
foreach line reduce [
"HELO"
rejoin ["MAIL FROM:<" return_email ">"]
rejoin ["RCPT TO:<" user_email ">"]
"DATA"
; ... etc.
][append m join line newline]
Anton.
[15/15] from: mattsmac:hot:mail at: 7-Nov-2003 10:29
Hi Brett,
Actually the EHLO liine is a valid command in extended SMTP. I am routing
this into our local exchange server, so it handles all the commands as they
come in. I know that the waits in the message are not a problem. I don't
use send because I needed to insert a line into the message preamble that
will signal that I need a read reply sent back by the mail client. All of
the code works fine on its own, when the file being attached is already
there. Its only when it is trying to access a file that was created
immediately prior that it has issues.
Again, this is not that big of a deal, as I found a work around using our
network file-sharing that is essentially just as effective.
Matt
-----------------------------------------
Hi Matt,
A few notes on your email.
1) Your HELO line is misspelt below to be EHLO.
2) Your code just blindly "dumps" the HELO along with MAIL FROM and RCPT TO
and the rest of the message down the line - it does not wait on and check
for server responses to these first three. With SMTP, HELO and QUIT could
take a little while to occur. On the HELO the SMTP server needs to acquire a
lock on the mailbox, on the QUIT it processes any deletions and releases the
lock (who knows what else it could be doing). So with multiple clients
coming in, you really need to wait to check that you get the lock.
3) Why not use SEND instead of your own code? It will wait on the HELO and
check the response. You can add custom headers in using the /HEADER
refinement of SEND.
4) Have you checked that the process your server code runs in has the
necessary privileges? Perhaps you should add some code to write a log to a
text file - to ensure that that it is doing what you hope it is doing.
Regards,
Brett.
Notes
- Quoted lines have been omitted from some messages.
View the message alone to see the lines that have been omitted