Welcome to the nuBuilder forums!

Please register and login to view forums and other content only available to registered users.

Give nuBuilder the ability to manage legacy data bases.

cypherinfo
Posts: 125
Joined: Thu Feb 18, 2010 5:42 pm
Contact:

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by cypherinfo »

Hello,


after an attempt to set an Activity - following as a general reference the tutorial about creating a Report - for a PHP procedure with internet explorer, I may not access to nuBuilder interface. After the login screen I get a blank page!

All works fine if I use another browser. No use to eliminate private data or so from the browser.

I tried to:

1 - create a form;
2 - add a new object on that form;
3 - add an activity with the PHP code as I suggested

Code: Select all

    do {
        mysql_query("UPDATE table SET field='{uniqid()}' WHERE field='' LIMIT 1");
    } while (mysql_affected_rows());
in order to fill an existing field with values nuBuilder is able to recognise and handles;
4 - run that acticity once.

I attached a video (download it, extract it before to watch it) to let you control all of the settings.
Do I guess a tutorial on how to add a nuBuilder primary in a not empty table is really needed.

Thank you.



Cypherinfo.
Attachments
nubuilder.7z
(677.98 KiB) Downloaded 428 times
cypherinfo
Posts: 125
Joined: Thu Feb 18, 2010 5:42 pm
Contact:

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by cypherinfo »

Hello,

I tried to fix it and I found that it takes long for running my PHP code with the Run activity feature of nuBuilder (on a table of only 17.000 records)!
I wonder if using your code may be different or not in terms of run time.

Thank you.

P.S: Anyway, the working code is:

Code: Select all

    do {
        $id = uniqid();
        mysql_query("UPDATE conservazione SET field='$id' WHERE field='' LIMIT 1");
       } while (mysql_affected_rows());
Cypherinfo.
steven
Posts: 218
Joined: Mon Jun 15, 2009 10:03 am
Has thanked: 1 time

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by steven »

Cypherinfo,

Speed shouldn't be an issue really because the process only needs to be run once.

If it works great!

(If it takes half an hour to run and you decide to spend half an hour on the code trying to make it faster, you really haven't saved anything)


Steven
If you like nuBuilder, how about leaving a nice review on SourceForge?
cypherinfo
Posts: 125
Joined: Thu Feb 18, 2010 5:42 pm
Contact:

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by cypherinfo »

Hello,


from the nuBuilder developer view I do guess speed is an issue; nuBuilder is for enterprises!
Another big issue that may boost nuBuilder spreading is to strength its ability to handle legacy data bases; I mean the capability to take an existing table (with its data and primary key) and put on it a nuBuilder interface without let the user knowing anything!

In order to summarize the key pro features of nuBuilder:
- designed for enterprise data base;
- fast;
- GPL;
- cross platform;

the cons are:
- it is not designed to handle already existing table (it is strange for an enterprise web application);
more, the design assumption is to manage empty tables or to create them together with its nuBuilder interface!

From my humble view:
1 - I need a web interface to embed and manage my not empty tables with my website;
2 - I run a search on the internet;
3 - I find nuBuilder;
4 - I try to use it with my tables;
5 - I notice it has to be edited in order to comply my tables;
6 - my tables may contains up to 16.000.000 records;
7 - I wonder: "is nuBuilder the right solution?";
8 - more, I wonder are there other potential users like me to move the nuBuilder developers to upgrade the actual release toward a fully legacy one?

I dream something like this:

An object or a screen where a user is asked to input: the table - keeping its primary key and data - to use to comply with nuBuilder and the rest is up to it.


Thank you.



Cypherinfo.
steven
Posts: 218
Joined: Mon Jun 15, 2009 10:03 am
Has thanked: 1 time

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by steven »

nuBuilder will never suit everone.
If you like nuBuilder, how about leaving a nice review on SourceForge?
cypherinfo
Posts: 125
Joined: Thu Feb 18, 2010 5:42 pm
Contact:

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by cypherinfo »

Well, it seems enterprise users are more and more demanding!

My review on you has been taked into consideration for the eweek newsletter as you may see here just for your being toward enterprises.

I wonder if you may consider to add the handling of the legacy data bases too as a must for nuBuilder now.



Thank you.



Cypherinfo.
Attachments
nuBuilder On Eweek.jpg
nuBuilder On Eweek.jpg (365.35 KiB) Viewed 12301 times
steven
Posts: 218
Joined: Mon Jun 15, 2009 10:03 am
Has thanked: 1 time

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by steven »

If you like nuBuilder, how about leaving a nice review on SourceForge?
christopher
Posts: 2
Joined: Mon Jun 15, 2009 5:03 am

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by christopher »

Hi Cipherinfo,

When working on legacy databases it is fairly common to have to make changes to your database if you want it to work in a new system. You cannot just expect the new system to handle your database as is, as every database has its own quirks.

The idea of a procedure to add necessary fields or modify them is a good one - though this would not require any change to nuBuilder itself but rather an additional procedure, and this is actually a task that would often be required in some form when shifting a database to a new platform. However, as jlcsusara has pointed out, it is possible to to change your field types so that they are compatible with nuBuilder while retaining the existing numeric fields, however any subsequent records created by nuBuilder would use its standard uniquid fields. As this is an open source system, users are encouraged to develop solutions and provide them back to the community to be included in modules or future releases.

Regarding the code you provided, another option is to do a mass update in batches by building a large update (UNTESTED - don't run this on your live database!), which may (or may not) be quicker:

Code: Select all

//This assumes you have an index on the new id field, conservazione_id, but it is not your primary key
//Set up base queries
$insertSQL = "INSERT INTO conservazione (old_id, conservazione_id) VALUES ";
$duplicateSQL = " ON DUPLICATE KEY UPDATE conservazione_id = VALUES(conservazione_id) ";
//if you are not familiar with ON DUPLICATE KEY UPDATE, it does an update if a key duplicate is found (i.e. every case in this instance)

$existingResultset = nuRunQuery("SELECT old_id FROM conservazione ORDER BY old_id");
$i = 0;
while($result = mysql_fetch_object($existingResultset)) {
    $newId = uniqid();
    if($valuesSQL != '') $valuesSQL .= ",";
    $valuesSQL .= "('{$result->old_id}','$newId')"; //Construct a mass update query
    if($i >= 1000) { //May be more efficient if higher or lower
        nuRunQuery($insertSQL.$valuesSQL.$duplicateSQL); //Run when it gets to a certain number of records to update
        $i = 0;
    }
    $i++;
}
//Finish off the last batch
if($i != 0)
	nuRunQuery($insertSQL.$valuesSQL.$duplicateSQL);
Another option is to create another table with the two columns, old_id (autonumber), new_id (VARCHAR(15)), populate with mass inserts:
"INSERT INTO id_temp_table (new_id) VALUES ('uniquid1'),('uniquid2'),('uniquid3'),('uniquid4')...('uniquid1000')", run enough times to match the records in your table.
Then "UPDATE conservazione INNER JOIN id_temp_table USING (old_id) SET conservazione_id = new_id;" (You may use LIMIT 10000 and WHERE conservazione IS NULL to do this in batches instead of all at once).
Once again, this assumes you have appropriate indexes on your tables, and I have not tested it, so it may or may not be the most efficient.

Bear in mind though, that when you are dealing with updates of millions of records, any system will take a while - that is why many reports and procedures are run in overnight batch jobs. But in one-off procedures like this, the time is less critical.

Or, you can just do what jlcsusara suggested.

Cheers,

Christopher
admin
Site Admin
Posts: 2781
Joined: Mon Jun 15, 2009 2:23 am
nuBuilder Version: 4.5
Been thanked: 1 time

Re: Give nuBuilder the ability to manage legacy data bases.

Unread post by admin »

.
Locked