World: r4wp
[Databases] group to discuss various database issues and drivers
older newer | first last |
afsanehsamim 16-Nov-2012 [329] | yes TomBon ,i did it ... but there are no values in my database. |
TomBon 16-Nov-2012 [330] | can you post your insert command here? |
afsanehsamim 16-Nov-2012 [331x3] | insert db ["insert into data1(oneone,onetwo,onethree,twoone,twothree,threeone,threetwo,threethree) values(?,?,?,?,?,?,?,?)" ] |
i know it dose not have any value | |
i do not know what should i write | |
BrianH 16-Nov-2012 [334] | You are missing the actual values to insert. Put those in the block after the SQL string. |
TomBon 16-Nov-2012 [335] | yes, I see. parameterized inserts are ok but perhaps better make a rejoin. |
Andreas 16-Nov-2012 [336] | insert db ["insert into sql-tablename (sql-fieldname) values (?)" cgi-values/cgi-fieldname] |
TomBon 16-Nov-2012 [337] | afsa, the last one from andreas is fine. |
BrianH 16-Nov-2012 [338] | TomBon, don't encourage people to use rejoin for SQL queries. Definitely use parameterized queries. Building your own queries with rejoin is a sure recipe for SQL injection. |
Andreas 16-Nov-2012 [339] | i suggest to get the html+cgi echoing working first, then getting a minimal script that inserts a value into your database working, and then putting the two pieces together by extending your "echo" cgi to insert into the database |
TomBon 16-Nov-2012 [340x2] | brian, made this for year without any problems. also good for beginners. |
checking for proper values and a corerct sql syntax should be always done even when parameterized. | |
BrianH 16-Nov-2012 [342x2] | Nice to hear, TomBon. Nonetheless, such checking is exactly what parameterized queries do, and I often have to fix errors made by other developers who don't use them. Plus, parameterized queries are a lot quicker on most databases because the query plan gets cached. |
It is always a bad idea to suggest to newbie programmers that they not use parameterized queries. | |
TomBon 16-Nov-2012 [344] | well better first to make him clear whats going up, then make the final. I think he is confused by this examples. btw, how parameterized queries preventing sql injection if not serverside? |
BrianH 16-Nov-2012 [345] | Non-parameterized queries are an advanced topic for experienced developers, though also the subject of the worst coding horror stories :) |
TomBon 16-Nov-2012 [346x2] | well, well :) |
but let's first try to help afsan, if his script is running he can improve it. | |
afsanehsamim 16-Nov-2012 [348x2] | guys ...i am happy :) it is working... tnx a lot Andreas ... |
thank you TomBon and BrianH | |
TomBon 16-Nov-2012 [350] | nice, good luck with your crossword afsan... |
BrianH 16-Nov-2012 [351] | With parameterized queries (even in REBOL) the SQL and the parameters are sent separately and combined in the server. The query plan is generated only once per query, with the parameter placeholders being in the plan. Then the actual parameters are plugged into the plan. The next time the parameterized query is called (maybe with differe3nt parameter values) the same plan is used and the new parameter values are plugged in. |
TomBon 16-Nov-2012 [352] | isn't this execution optimation?. in this case a stored procedure will help also. how will this prevent from sql injection? compared to a concatenated server side sql string? |
BrianH 16-Nov-2012 [353x2] | If you build a query dynamically with rejoin or something, the query is put together client side and then the server has to generate a new query plan for each distinct set of parameter values. This takes time and blows the query plan cache, which slows down the whole query process. |
The problem is that your ad-hoc parameter screening is usually not perfect. Parameterized queries don't build a query in the server, they just plug in the values to an already-compiled query (the "query plan"). The server doesn't have to do any parameter screening other than for malformed values in the protocol. | |
TomBon 16-Nov-2012 [355] | depends on the needs. I always try to detach the data sink from input logic. this way you can change your db backend very easy but of course everybody has it's own style in this. |
BrianH 16-Nov-2012 [356x2] | For new developers ad-hoc parameter screening is even more likely to be bad (and most that don't use parameterized queries are still new, no matter how long they've been programming, because parameterized queries are almost always inherently better). Even if it wasn't a safety issue, they're a lot faster. |
I've seen data front-ends that don't use parametrized queries when talking to SQL servers that support them. They need work. | |
afsanehsamim 16-Nov-2012 [358] | could you tell me how can i compare values of two tables in database? |
Endo 17-Nov-2012 [359] | About parametrized queries: The only problem using them on R2, at least with RT's default ODBC, there is no chance to use NULL values. None of those work: insert db-port ["INSERT t (a) VALUES (?)" NULL] insert db-port ["INSERT t (a) VALUES (?)" 'NULL] insert db-port ["INSERT t (a) VALUES (?)" "NULL"] insert db-port ["INSERT t (a) VALUES (?)" none] insert db-port reduce ["INSERT t (a) VALUES (?)" none] |
TomBon 17-Nov-2012 [360] | you have more than one solution, the first is a simple serial SELECT on each table -> compare the output for equal. of course this produce unnecessary DB overhead but I guess you won't feel any speed difference except you are serving a whole city concurrently. another, better one is a JOIN or UNION. SELECT table_name1.column_name(s), ... FROM table_name1 LEFT JOIN table_name2 ON table_name1.column_name=table_name2.column_name the JOIN direction (LEFT,RIGHT,INNER) for your reference table is important here. the resultset is a table containing BOTH columns. if both having a value -> match, if one is empty then you don't. index both fields to accelerate the query and use something like the free SQLyog to test different queries to make debugging easier for you. while you situation reminds me to myself, sitting infront of a monochrom asthon tate dot some decades ago and asking what next?, you should 'bite' yourself now thru the rest. It won't help you on longterm if you don't. |
afsanehsamim 17-Nov-2012 [361x11] | @TomBon: my query for joining two tables is :insert db["select * from data LEFT JOIN data1 ON data.oneone=data1.oneone"] and output is :[ ["c" "a" "t" "a" "e" "r" "o" "a" none none none none none none none none] ] plz tell me what should i write in query that i get values instead of none in output ? |
guys when i enter correct value in form the above join query works properly... i need help for writing queries which other condition,it means if user enter wrong value ,it joins with first table but dose comparing indicidually and shows error message. | |
the output of this query insert db[{select * from data LEFT JOIN data1 ON data.oneone=data1.oneone}] is : [ ["c" "a" "t" "a" "e" "r" "o" "a" "c" "a" "t" "a" "e" "r" "o" "a"] ] | |
is there anyone who can help me ?? | |
i compare each field of tables with each other like :insert db["select data.oneone,data1.oneone from data LEFT JOIN data1 ON data.oneone=data1.oneone"] results: copy db probe results insert db["select data.onetwo,data1.onetwo from data LEFT JOIN data1 ON data.onetwo=data1.onetwo"] results: copy db probe results insert db["select data.onethree,data1.onethree from data LEFT JOIN data1 ON data.onethree=data1.onethree"] results: copy db probe results ..... | |
i got result | |
i need codes for showing message to user ,it mean after each joining ,it should show user that value is correct or no | |
guys ! could you plz tell me after comparing values of two tables how can we show the output on web page? | |
after writing queries :foreach row read/custom mysql://[root-:-localhost]/test ["select data.oneone,data1.oneone from data LEFT JOIN data1 ON data.oneone=data1.oneone"] [print row] foreach row read/custom mysql://[root-:-localhost]/test ["select data.onetwo,data1.onetwo from data LEFT JOIN data1 ON data.onetwo=data1.onetwo"] [print row] .... | |
i got this results:c c a none t t a none e none r none o none a none | |
now how can i write query for everyvalues which are same and print correct message on web page? | |
afsanehsamim 23-Nov-2012 [372x2] | hey guys... i have just 2days time for my project ! could you help me? |
i could not do the last step ... i should show result of comparing values on web page | |
TomBon 11-Dec-2012 [374] | a quick update on elasticsearch. Currently I have reached 2TB datasize (~85M documents) on a single node. Queries now starting to slow down but the system is very stable even under heavy load. While queries in average took between 50-250ms against a dataset around 1TB the same queries are now in a range between 900-1500 ms. The average allocated java heap is around 9GB which is nearly 100% of the max heap size by a 15 shards and 0 replicas setting. elasticsearch looks like a very good candidate for handling big data with a need for 'near realtime' analysis. Classical RDBMS like mysql and postgresql where grilled at around 150-500GB. Another tested candidate was MongoDB which was great too but since it stores all metadata and fields uncompressed the waste of diskspace was ridiculous high. Furthermore query execution times differs unexpectable without any known reason by factor 3. Tokyo Cabinet started fine but around 1TB I have noticed file integrity problems which leads into endless restoring/repairing procedures. Adding sharding logic by coding an additional layer wasn't very motivating but could solve this issue. Within the next six months the datasize should reached the 100TB mark. Would be interesting to see how elasticsearch will scale and how many nodes are nessesary to handle this efficiently. |
Maxim 11-Dec-2012 [375] | when you talk about "documents" what type of documents are they? |
Gregg 11-Dec-2012 [376] | Thanks for the info Tomas. |
TomBon 12-Dec-2012 [377] | crawled html/mime embedded documents/images etc. as plain compressed source (avg. 25kb) and 14 searchable metafields (ngram) to train different NN types for pattern recognition. |
Maxim 12-Dec-2012 [378] | thanks :-) |
older newer | first last |