Subject: Re: [MTT users] MTT server side problem
From: Pavel Shamis (Pasha) (pasha_at_[hidden])
Date: 2008-05-20 10:05:15
Josh Hursey wrote:
> I just got back from travel this morning. I reviewed the patch and it
> looks good to me.
>
> I updated the ticket:
> https://svn.open-mpi.org/trac/mtt/ticket/357
>
> Do you need me to commit it?
Thank you. I will commit it.
Regards,
Pasha
>
> Cheers,
> Josh
>
> On May 19, 2008, at 7:32 AM, Pavel Shamis (Pasha) wrote:
>
>> Hello,
>> Did you have chance to review this patch ?
>>
>> Regards,
>> Pasha
>>
>> Josh Hursey wrote:
>>> Sorry for the delay on this. I probably will not have a chance to
>>> look at it until later this week or early next. Thank you for the
>>> work on the patch.
>>>
>>> Cheers,
>>> Josh
>>>
>>> On May 12, 2008, at 8:08 AM, Pavel Shamis (Pasha) wrote:
>>>
>>>> Hi Josh,
>>>> I ported the error handling mechanism from submit/index.php to to
>>>> the database.inc. Please review.
>>>>
>>>> Thanks,
>>>> Pasha
>>>>
>>>> Josh Hursey wrote:
>>>>> Pasha,
>>>>>
>>>>> I'm looking at the patch a bit closer and even though at a high
>>>>> level the do_pg_connect, do_pg_query, simple_select, and select
>>>>> functions do the same thing the versions in submit/index.php have
>>>>> some additional error handling mechanisms that the ones in
>>>>> database.inc do not have. Specifically they send email when the
>>>>> functions fail with messages indicating what failed so corrections
>>>>> can be made.
>>>>>
>>>>> So though I agree that we should unify the functionality I cannot
>>>>> recommend this patch since it will result in losing useful error
>>>>> handling functionality. Maybe there is another way to clean this
>>>>> up to preserve the error reporting.
>>>>>
>>>>> -- Josh
>>>>>
>>>>> On May 7, 2008, at 11:56 AM, Pavel Shamis (Pasha) wrote:
>>>>>
>>>>>> Hi Josh,
>>>>>> I had the original problem with some old revision from trunk.
>>>>>> Today I updated the server to latest revision from trunk + the
>>>>>> patch and everything looks good.
>>>>>>
>>>>>> Can I commit the patch ?
>>>>>>
>>>>>> Pasha
>>>>>>
>>>>>>
>>>>>> Ethan Mallove wrote:
>>>>>>> On Wed, May/07/2008 06:04:07PM, Pavel Shamis (Pasha) wrote:
>>>>>>>
>>>>>>>> Hi Josh.
>>>>>>>>
>>>>>>>>> Looking at the patch I'm a little bit conserned. The
>>>>>>>>> "get_table_fields()" is, as you mentioned, no longer used so
>>>>>>>>> should be removed. However the other functions are critical to
>>>>>>>>> the submission script particularly 'do_pg_connect' which opens
>>>>>>>>> the connection to the backend database.
>>>>>>>>>
>>>>>>>> All the functions are implemented in $topdir/database.inc
>>>>>>>> file. And the "database.inc" implementation is better because
>>>>>>>> it use password and username from config.ini. The original
>>>>>>>> implementation from submit/index use
>>>>>>>> hardcoded values defined in the file.
>>>>>>>>
>>>>>>>>> Are you using the current development trunk (mtt/trunk) or the
>>>>>>>>> stable release branch (mtt/branches/ompi-core-testers)?
>>>>>>>>>
>>>>>>>> trunk
>>>>>>>>
>>>>>>>>> Can you send us the error messages that you were receiving?
>>>>>>>>>
>>>>>>>> 1. On client side I see ""*** WARNING: MTTDatabase client did
>>>>>>>> not get a serial"
>>>>>>>> As result of the error some of MTT results is not visible via
>>>>>>>> the web reporter
>>>>>>>> 2. On server side I found follow error message:
>>>>>>>> [client 10.4.3.214] PHP Fatal error: Allowed memory size of
>>>>>>>> 33554432 bytes exhausted (tried to allocate 23592960
>>>>>>>> bytes) in /.autodirect/swgwork/MTT/mtt/submit/index.php(79) :
>>>>>>>> eval()'d code on line 77515
>>>>>>>> [Mon May 05 19:26:05 2008] [notice] caught SIGTERM, shutting down
>>>>>>>> [Mon May 05 19:30:54 2008] [notice] suEXEC mechanism enabled
>>>>>>>> (wrapper: /usr/sbin/suexec)
>>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: generating secret
>>>>>>>> for digest authentication ...
>>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: done
>>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: Built with OpenLDAP
>>>>>>>> LDAP SDK
>>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: SSL support unavailable
>>>>>>>> My memory limit in php.ini file was set on 256MB !
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Looks like PHP is actually using a 32MB limit ("Allowed
>>>>>>> memory size of 33554432 ..."). Does a (Apache?) daemon need
>>>>>>> to be restarted for the php.ini file to take effect? To
>>>>>>> check your settings, this little PHP script will print an
>>>>>>> HTML page of all the active system settings (search on
>>>>>>> "memory_limit").
>>>>>>> <?php
>>>>>>> phpinfo();
>>>>>>> ?>
>>>>>>>
>>>>>>> -Ethan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Pasha
>>>>>>>>
>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Josh
>>>>>>>>>
>>>>>>>>> On May 7, 2008, at 4:49 AM, Pavel Shamis (Pasha) wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I upgraded the server side (the mtt is still running , so
>>>>>>>>>> don't know if the problem was resolved)
>>>>>>>>>> During upgrade I had some problem with the submit/index.php
>>>>>>>>>> script, it had some duplicated functions and some of them
>>>>>>>>>> were broken.
>>>>>>>>>> Please review the attached patch.
>>>>>>>>>>
>>>>>>>>>> Pasha
>>>>>>>>>>
>>>>>>>>>> Ethan Mallove wrote:
>>>>>>>>>>
>>>>>>>>>>> On Tue, May/06/2008 06:29:33PM, Pavel Shamis (Pasha) wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>> I'm not sure which cron jobs you're referring to. Do you
>>>>>>>>>>>>> mean these?
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://svn.open-mpi.org/trac/mtt/browser/trunk/server/php/cron
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> I talked about this one:
>>>>>>>>>>>> https://svn.open-mpi.org/trac/mtt/wiki/ServerMaintenance
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> I'm guessing you would only be concerned with the below
>>>>>>>>>>> periodic-maintenance.pl script, which just runs
>>>>>>>>>>> ANALYZE/VACUUM queries. I think you can start that up
>>>>>>>>>>> whenever you want (and it should optimize the Reporter).
>>>>>>>>>>>
>>>>>>>>>>> https://svn.open-mpi.org/trac/mtt/browser/trunk/server/sql/cron/periodic-maintenance.pl
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> -Ethan
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>> The only thing there are the regular
>>>>>>>>>>>>> mtt-results_at_[hidden] email alerts and some out-of-date
>>>>>>>>>>>>> DB monitoring junk. You can ignore that stuff.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Josh, are there some nightly (DB
>>>>>>>>>>>>> pruning/cleaning/vacuuming?) cron jobs that Pasha should be
>>>>>>>>>>>>> running?
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Ethan
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ethan Mallove wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Pasha,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I thought this issue was solved in r1119 (see below). Do
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> have the latest mtt/server scripts?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://svn.open-mpi.org/trac/mtt/changeset/1119/trunk/server/php/submit
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Ethan
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, May/06/2008 03:26:43PM, Pavel Shamis (Pasha) wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> About the issue:
>>>>>>>>>>>>>>>> 1. On client side I see ""*** WARNING: MTTDatabase
>>>>>>>>>>>>>>>> client did not get a serial"
>>>>>>>>>>>>>>>> As result of the error some of MTT results is not
>>>>>>>>>>>>>>>> visible via the web reporter
>>>>>>>>>>>>>>>> 2. On server side I found follow error message:
>>>>>>>>>>>>>>>> [client 10.4.3.214] PHP Fatal error: Allowed memory
>>>>>>>>>>>>>>>> size of 33554432 bytes exhausted (tried to allocate
>>>>>>>>>>>>>>>> 23592960
>>>>>>>>>>>>>>>> bytes) in
>>>>>>>>>>>>>>>> /.autodirect/swgwork/MTT/mtt/submit/index.php(79) :
>>>>>>>>>>>>>>>> eval()'d code on line 77515
>>>>>>>>>>>>>>>> [Mon May 05 19:26:05 2008] [notice] caught SIGTERM,
>>>>>>>>>>>>>>>> shutting down
>>>>>>>>>>>>>>>> [Mon May 05 19:30:54 2008] [notice] suEXEC mechanism
>>>>>>>>>>>>>>>> enabled (wrapper: /usr/sbin/suexec)
>>>>>>>>>>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: generating
>>>>>>>>>>>>>>>> secret for digest authentication ...
>>>>>>>>>>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: done
>>>>>>>>>>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: Built with
>>>>>>>>>>>>>>>> OpenLDAP LDAP SDK
>>>>>>>>>>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: SSL support
>>>>>>>>>>>>>>>> unavailable
>>>>>>>>>>>>>>>> My memory limit in php.ini file was set on 256MB !
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any ideas ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Pavel Shamis (Pasha)
>>>>>>>>>>>>>>>> Mellanox Technologies
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> mtt-users mailing list
>>>>>>>>>>>>>>>> mtt-users_at_[hidden]
>>>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Pavel Shamis (Pasha)
>>>>>>>>>>>>>> Mellanox Technologies
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> mtt-users mailing list
>>>>>>>>>>>>>> mtt-users_at_[hidden]
>>>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Pavel Shamis (Pasha)
>>>>>>>>>>>> Mellanox Technologies
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> mtt-users mailing list
>>>>>>>>>>>> mtt-users_at_[hidden]
>>>>>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Pavel Shamis (Pasha)
>>>>>>>>>> Mellanox Technologies
>>>>>>>>>>
>>>>>>>>>> Index: submit/index.php
>>>>>>>>>> ===================================================================
>>>>>>>>>>
>>>>>>>>>> --- submit/index.php (revision 1200)
>>>>>>>>>> +++ submit/index.php (working copy)
>>>>>>>>>> @@ -1,6 +1,7 @@
>>>>>>>>>> <?php
>>>>>>>>>> # Copyright (c) 2006 Cisco Systems, Inc. All rights
>>>>>>>>>> reserved.
>>>>>>>>>> # Copyright (c) 2006-2007 Sun Microsystems, Inc. All rights
>>>>>>>>>> reserved.
>>>>>>>>>> +# Copyright (c) 2008 Mellanox Technologies. All rights
>>>>>>>>>> reserved.
>>>>>>>>>>
>>>>>>>>>> #
>>>>>>>>>> #
>>>>>>>>>> @@ -24,8 +25,7 @@ if (file_exists("$topdir/config.inc")) {
>>>>>>>>>> ini_set("memory_limit", "32M");
>>>>>>>>>>
>>>>>>>>>> $topdir = '..';
>>>>>>>>>> -$ompi_home = '/l/osl/www/doc/www.open-mpi.org';
>>>>>>>>>> -include_once("$ompi_home/dbpassword.inc");
>>>>>>>>>> +include_once("$topdir/database.inc");
>>>>>>>>>> include_once("$topdir/reporter.inc");
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> @@ -1465,60 +1465,6 @@ function get_table_indexes($table_name,
>>>>>>>>>> return simple_select($sql_cmd);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> -# Function used to determine which _POST fields
>>>>>>>>>> -# to INSERT. Prevent non-existent fields from being
>>>>>>>>>> -# INSERTed
>>>>>>>>>> -function get_table_fields($table_name) {
>>>>>>>>>> -
>>>>>>>>>> - global $dbname;
>>>>>>>>>> - global $id;
>>>>>>>>>> -
>>>>>>>>>> - # These indexes are special in that they link phases
>>>>>>>>>> - # together and hence, can and do show up in _POST
>>>>>>>>>> - if ($table_name == "test_build")
>>>>>>>>>> - $special_indexes = array("mpi_install$id");
>>>>>>>>>> - elseif ($table_name == "test_run")
>>>>>>>>>> - $special_indexes = array("test_build$id");
>>>>>>>>>> -
>>>>>>>>>> - # Crude way to tell whether a field is an index
>>>>>>>>>> - $is_not_index_clause =
>>>>>>>>>> - "\n\t (table_name = '$table_name' AND NOT " .
>>>>>>>>>> - "\n\t (data_type = 'integer' AND " .
>>>>>>>>>> - "\n\t column_name ~ '_id$' AND " .
>>>>>>>>>> - "\n\t table_catalog = '$dbname'))";
>>>>>>>>>> -
>>>>>>>>>> - $is_special_index_clause =
>>>>>>>>>> - "\n\t (table_name = '$table_name' AND " .
>>>>>>>>>> - "\n\t (column_name = '$special_indexes[0]'))";
>>>>>>>>>> -
>>>>>>>>>> - $is_index_columns = array(
>>>>>>>>>> - "column_name",
>>>>>>>>>> - "data_type",
>>>>>>>>>> - "column_default");
>>>>>>>>>> -
>>>>>>>>>> - $sql_cmd = "\n SELECT " . join(",",$is_index_columns) .
>>>>>>>>>> - "\n\t FROM information_schema.columns WHERE " .
>>>>>>>>>> - "\n\t " .
>>>>>>>>>> - $is_not_index_clause . " OR " .
>>>>>>>>>> - $is_special_index_clause . ';';
>>>>>>>>>> -
>>>>>>>>>> - do_pg_connect();
>>>>>>>>>> -
>>>>>>>>>> - # This table will be easier to manage if it's
>>>>>>>>>> - # keyed by column, instead of index
>>>>>>>>>> - $tmp = array();
>>>>>>>>>> - $arr = array();
>>>>>>>>>> - $arr = select($sql_cmd);
>>>>>>>>>> -
>>>>>>>>>> - foreach ($is_index_columns as $col) {
>>>>>>>>>> - $tmp[$col] = array();
>>>>>>>>>> - for ($i = 0; $i < sizeof($arr); $i++) {
>>>>>>>>>> - $tmp[$col][] = $arr[$i][$col];
>>>>>>>>>> - }
>>>>>>>>>> - }
>>>>>>>>>> - return $tmp;
>>>>>>>>>> -}
>>>>>>>>>> -
>>>>>>>>>> #
>>>>>>>>>> # Useful display of all the parameters posted.
>>>>>>>>>> # Warning: this function could explode the memory footprint
>>>>>>>>>> causing the
>>>>>>>>>> @@ -1622,90 +1568,6 @@ function is_null_($var) {
>>>>>>>>>>
>>>>>>>>>> ######################################################################
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -function do_pg_connect() {
>>>>>>>>>> -
>>>>>>>>>> - global $dbname;
>>>>>>>>>> - global $user;
>>>>>>>>>> - global $pass;
>>>>>>>>>> - global $pgsql_conn;
>>>>>>>>>> - static $connected = false;
>>>>>>>>>> -
>>>>>>>>>> - if (!$connected) {
>>>>>>>>>> - $pgsql_conn =
>>>>>>>>>> - pg_connect("host=localhost port=5432
>>>>>>>>>> dbname=$dbname user=$user password=$pass");
>>>>>>>>>> -
>>>>>>>>>> - # Exit if we cannot connect
>>>>>>>>>> - if (!$pgsql_conn) {
>>>>>>>>>> - mtt_abort("\nCould not connect to the $dbname
>>>>>>>>>> database; " .
>>>>>>>>>> - "submit this run later.");
>>>>>>>>>> - }
>>>>>>>>>> - else {
>>>>>>>>>> - $connected = true;
>>>>>>>>>> -
>>>>>>>>>> - # Serialize all transactions by default - Safer
>>>>>>>>>> that way.
>>>>>>>>>> - do_pg_query("SET SESSION CHARACTERISTICS AS
>>>>>>>>>> TRANSACTION ISOLATION LEVEL SERIALIZABLE", false);
>>>>>>>>>> - }
>>>>>>>>>> -
>>>>>>>>>> - # pg_set_error_verbosity($pgsql_conn,
>>>>>>>>>> PGSQL_ERRORS_VERBOSE); # PHP 5 needed
>>>>>>>>>> - # pg_trace($_ENV['HOME'] . "/pgsql.trace", 'w',
>>>>>>>>>> $pgsql_conn);
>>>>>>>>>> - # debug("\npg_options: " .
>>>>>>>>>> var_export(pg_options($pgsql_conn)));
>>>>>>>>>> - }
>>>>>>>>>> -}
>>>>>>>>>> -
>>>>>>>>>> -function do_pg_query($cmd, $silent) {
>>>>>>>>>> - $db_res = null;
>>>>>>>>>> -
>>>>>>>>>> - do_pg_connect();
>>>>>>>>>> -
>>>>>>>>>> - debug("\nSQL: $cmd\n");
>>>>>>>>>> - if (! ($db_res = pg_query($cmd))) {
>>>>>>>>>> - $out = "\nSQL QUERY: " . $cmd .
>>>>>>>>>> - "\nSQL ERROR: " . pg_last_error() .
>>>>>>>>>> - "\nSQL ERROR: " . pg_result_error();
>>>>>>>>>> -
>>>>>>>>>> - # Some errors are unsurprising, allow for silence in
>>>>>>>>>> - # such cases
>>>>>>>>>> - if (! $silent) {
>>>>>>>>>> - mtt_error($out);
>>>>>>>>>> - mtt_send_mail($out);
>>>>>>>>>> - }
>>>>>>>>>> - }
>>>>>>>>>> - debug("\nDatabase rows affected: " .
>>>>>>>>>> pg_affected_rows($db_res) . "\n");
>>>>>>>>>> -
>>>>>>>>>> - return $db_res;
>>>>>>>>>> -}
>>>>>>>>>> -
>>>>>>>>>> -# Fetch scalar value
>>>>>>>>>> -function select_scalar($cmd) {
>>>>>>>>>> -
>>>>>>>>>> - $set = array();
>>>>>>>>>> - $set = simple_select($cmd);
>>>>>>>>>> - return array_shift($set);
>>>>>>>>>> -}
>>>>>>>>>> -
>>>>>>>>>> -# Fetch 1D array
>>>>>>>>>> -function simple_select($cmd) {
>>>>>>>>>> -
>>>>>>>>>> - do_pg_connect();
>>>>>>>>>> -
>>>>>>>>>> - $rows = null;
>>>>>>>>>> -
>>>>>>>>>> - debug("\nSQL: $cmd\n");
>>>>>>>>>> - if (! ($result = pg_query($cmd))) {
>>>>>>>>>> - $out = "\nSQL QUERY: " . $cmd .
>>>>>>>>>> - "\nSQL ERROR: " . pg_last_error() .
>>>>>>>>>> - "\nSQL ERROR: " . pg_result_error();
>>>>>>>>>> - mtt_error($out);
>>>>>>>>>> - mtt_send_mail($out);
>>>>>>>>>> - }
>>>>>>>>>> - $max = pg_num_rows($result);
>>>>>>>>>> - for ($i = 0; $i < $max; ++$i) {
>>>>>>>>>> - $row = pg_fetch_array($result, $i, PGSQL_NUM);
>>>>>>>>>> - $rows[] = $row[0];
>>>>>>>>>> - }
>>>>>>>>>> - return $rows;
>>>>>>>>>> -}
>>>>>>>>>> -
>>>>>>>>>> # Fetch an associative hash (column name => value)
>>>>>>>>>> function associative_select($cmd) {
>>>>>>>>>>
>>>>>>>>>> @@ -1722,21 +1584,6 @@ function associative_select($cmd) {
>>>>>>>>>> return pg_fetch_array($result);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> -# Fetch 2D array
>>>>>>>>>> -function select($cmd) {
>>>>>>>>>> - do_pg_connect();
>>>>>>>>>> -
>>>>>>>>>> - debug("\nSQL: $cmd\n");
>>>>>>>>>> - if (! ($result = pg_query($cmd))) {
>>>>>>>>>> - $out = "\nSQL QUERY: " . $cmd .
>>>>>>>>>> - "\nSQL ERROR: " . pg_last_error() .
>>>>>>>>>> - "\nSQL ERROR: " . pg_result_error();
>>>>>>>>>> - mtt_error($out);
>>>>>>>>>> - mtt_send_mail($out);
>>>>>>>>>> - }
>>>>>>>>>> - return pg_fetch_all($result);
>>>>>>>>>> -}
>>>>>>>>>> -
>>>>>>>>>> ######################################################################
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> # Function for reporting errors back to the client
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Pavel Shamis (Pasha)
>>>>>>>> Mellanox Technologies
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> mtt-users mailing list
>>>>>>>> mtt-users_at_[hidden]
>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Pavel Shamis (Pasha)
>>>>>> Mellanox Technologies
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Pavel Shamis (Pasha)
>>>> Mellanox Technologies
>>>>
>>>> Index: database.inc
>>>> ===================================================================
>>>> --- database.inc (revision 1200)
>>>> +++ database.inc (working copy)
>>>> @@ -3,6 +3,7 @@
>>>> #
>>>> # Copyright (c) 2006 Sun Microsystems, Inc.
>>>> # All rights reserved.
>>>> +# Copyright (c) 2008 Mellanox Technologies. All rights
>>>> reserved.
>>>> # $COPYRIGHT$
>>>> #
>>>> # Additional copyrights may follow
>>>> @@ -100,10 +101,13 @@ function do_pg_connect() {
>>>> $pgsql_conn =
>>>> pg_connect("host=localhost port=5432
>>>> dbname=$mtt_database_name user=$mtt_database_username
>>>> password=$mtt_database_password");
>>>> if (!$pgsql_conn) {
>>>> - print("\nCould not connect to '$dbname' database.");
>>>> - exit;
>>>> + mtt_abort("\nCould not connect to the $dbname
>>>> database; " .
>>>> + "submit this run later.");
>>>> + } else {
>>>> + $connected = true;
>>>> +# Serialize all transactions by default - Safer that way.
>>>> + do_pg_query("SET SESSION CHARACTERISTICS AS
>>>> TRANSACTION ISOLATION LEVEL SERIALIZABLE", false);
>>>> }
>>>> - $connected = true;
>>>> }
>>>> }
>>>>
>>>> @@ -113,12 +117,17 @@ function do_pg_query($cmd, $silent) {
>>>>
>>>> $start = time();
>>>>
>>>> - debug_sql($cmd, 2);
>>>> + debug("\nSQL: $cmd\n");
>>>> if (! ($db_res = pg_query($cmd))) {
>>>> + $out = "\nSQL QUERY: " . $cmd .
>>>> + "\nSQL ERROR: " . pg_last_error() .
>>>> + "\nSQL ERROR: " . pg_result_error();
>>>> +
>>>> +# Some errors are unsurprising, allow for silence in
>>>> +# such cases
>>>> if (! $silent) {
>>>> - print("\npostgres: " .
>>>> - pg_last_error() . "\n" .
>>>> - pg_result_error());
>>>> + mtt_error($out);
>>>> + mtt_send_mail($out);
>>>> }
>>>> }
>>>>
>>>> @@ -137,21 +146,9 @@ function do_pg_query($cmd, $silent) {
>>>>
>>>> # Fetch scalar value
>>>> function select_scalar($cmd) {
>>>> - do_pg_connect();
>>>> -
>>>> - $rows = null;
>>>> -
>>>> - debug_sql($cmd, 2);
>>>> - if (! ($result = pg_query($cmd))) {
>>>> - print("\npostgres: " . pg_last_error() . "\n" .
>>>> - pg_result_error());
>>>> - }
>>>> - $max = pg_num_rows($result);
>>>> - for ($i = 0; $i < $max; ++$i) {
>>>> - $row = pg_fetch_array($result, $i, PGSQL_NUM);
>>>> - $rows[] = $row[0];
>>>> - }
>>>> - return array_shift($rows);
>>>> + $set = array();
>>>> + $set = simple_select($cmd);
>>>> + return array_shift($set);
>>>> }
>>>>
>>>> # Fetch 1D array
>>>> @@ -160,10 +157,13 @@ function simple_select($cmd) {
>>>>
>>>> $rows = null;
>>>>
>>>> - debug_sql($cmd, 2);
>>>> + debug("\nSQL: $cmd\n");
>>>> if (! ($result = pg_query($cmd))) {
>>>> - print("\npostgres: " . pg_last_error() . "\n" .
>>>> - pg_result_error());
>>>> + $out = "\nSQL QUERY: " . $cmd .
>>>> + "\nSQL ERROR: " . pg_last_error() .
>>>> + "\nSQL ERROR: " . pg_result_error();
>>>> + mtt_error($out);
>>>> + mtt_send_mail($out);
>>>> }
>>>> $max = pg_num_rows($result);
>>>> for ($i = 0; $i < $max; ++$i) {
>>>> @@ -177,10 +177,13 @@ function simple_select($cmd) {
>>>> function select($cmd) {
>>>> do_pg_connect();
>>>>
>>>> - debug_sql($cmd, 2);
>>>> + debug("\nSQL: $cmd\n");
>>>> if (! ($result = pg_query($cmd))) {
>>>> - print("\npostgres: " . pg_last_error() . "\n" .
>>>> - pg_result_error());
>>>> + $out = "\nSQL QUERY: " . $cmd .
>>>> + "\nSQL ERROR: " . pg_last_error() .
>>>> + "\nSQL ERROR: " . pg_result_error();
>>>> + mtt_error($out);
>>>> + mtt_send_mail($out);
>>>> }
>>>> return pg_fetch_all($result);
>>>> }
>>>> Index: submit/index.php
>>>> ===================================================================
>>>> --- submit/index.php (revision 1200)
>>>> +++ submit/index.php (working copy)
>>>> @@ -1,6 +1,7 @@
>>>> <?php
>>>> # Copyright (c) 2006 Cisco Systems, Inc. All rights reserved.
>>>> # Copyright (c) 2006-2007 Sun Microsystems, Inc. All rights reserved.
>>>> +# Copyright (c) 2008 Mellanox Technologies. All rights
>>>> reserved.
>>>>
>>>> #
>>>> #
>>>> @@ -24,8 +25,7 @@ if (file_exists("$topdir/config.inc")) {
>>>> ini_set("memory_limit", "32M");
>>>>
>>>> $topdir = '..';
>>>> -$ompi_home = '/l/osl/www/doc/www.open-mpi.org';
>>>> -include_once("$ompi_home/dbpassword.inc");
>>>> +include_once("$topdir/database.inc");
>>>> include_once("$topdir/reporter.inc");
>>>>
>>>>
>>>> @@ -1465,60 +1465,6 @@ function get_table_indexes($table_name,
>>>> return simple_select($sql_cmd);
>>>> }
>>>>
>>>> -# Function used to determine which _POST fields
>>>> -# to INSERT. Prevent non-existent fields from being
>>>> -# INSERTed
>>>> -function get_table_fields($table_name) {
>>>> -
>>>> - global $dbname;
>>>> - global $id;
>>>> -
>>>> - # These indexes are special in that they link phases
>>>> - # together and hence, can and do show up in _POST
>>>> - if ($table_name == "test_build")
>>>> - $special_indexes = array("mpi_install$id");
>>>> - elseif ($table_name == "test_run")
>>>> - $special_indexes = array("test_build$id");
>>>> -
>>>> - # Crude way to tell whether a field is an index
>>>> - $is_not_index_clause =
>>>> - "\n\t (table_name = '$table_name' AND NOT " .
>>>> - "\n\t (data_type = 'integer' AND " .
>>>> - "\n\t column_name ~ '_id$' AND " .
>>>> - "\n\t table_catalog = '$dbname'))";
>>>> -
>>>> - $is_special_index_clause =
>>>> - "\n\t (table_name = '$table_name' AND " .
>>>> - "\n\t (column_name = '$special_indexes[0]'))";
>>>> -
>>>> - $is_index_columns = array(
>>>> - "column_name",
>>>> - "data_type",
>>>> - "column_default");
>>>> -
>>>> - $sql_cmd = "\n SELECT " . join(",",$is_index_columns) .
>>>> - "\n\t FROM information_schema.columns WHERE " .
>>>> - "\n\t " .
>>>> - $is_not_index_clause . " OR " .
>>>> - $is_special_index_clause . ';';
>>>> -
>>>> - do_pg_connect();
>>>> -
>>>> - # This table will be easier to manage if it's
>>>> - # keyed by column, instead of index
>>>> - $tmp = array();
>>>> - $arr = array();
>>>> - $arr = select($sql_cmd);
>>>> -
>>>> - foreach ($is_index_columns as $col) {
>>>> - $tmp[$col] = array();
>>>> - for ($i = 0; $i < sizeof($arr); $i++) {
>>>> - $tmp[$col][] = $arr[$i][$col];
>>>> - }
>>>> - }
>>>> - return $tmp;
>>>> -}
>>>> -
>>>> #
>>>> # Useful display of all the parameters posted.
>>>> # Warning: this function could explode the memory footprint causing
>>>> the
>>>> @@ -1622,90 +1568,6 @@ function is_null_($var) {
>>>>
>>>> ######################################################################
>>>>
>>>> -function do_pg_connect() {
>>>> -
>>>> - global $dbname;
>>>> - global $user;
>>>> - global $pass;
>>>> - global $pgsql_conn;
>>>> - static $connected = false;
>>>> -
>>>> - if (!$connected) {
>>>> - $pgsql_conn =
>>>> - pg_connect("host=localhost port=5432 dbname=$dbname
>>>> user=$user password=$pass");
>>>> -
>>>> - # Exit if we cannot connect
>>>> - if (!$pgsql_conn) {
>>>> - mtt_abort("\nCould not connect to the $dbname
>>>> database; " .
>>>> - "submit this run later.");
>>>> - }
>>>> - else {
>>>> - $connected = true;
>>>> -
>>>> - # Serialize all transactions by default - Safer that way.
>>>> - do_pg_query("SET SESSION CHARACTERISTICS AS
>>>> TRANSACTION ISOLATION LEVEL SERIALIZABLE", false);
>>>> - }
>>>> -
>>>> - # pg_set_error_verbosity($pgsql_conn,
>>>> PGSQL_ERRORS_VERBOSE); # PHP 5 needed
>>>> - # pg_trace($_ENV['HOME'] . "/pgsql.trace", 'w', $pgsql_conn);
>>>> - # debug("\npg_options: " .
>>>> var_export(pg_options($pgsql_conn)));
>>>> - }
>>>> -}
>>>> -
>>>> -function do_pg_query($cmd, $silent) {
>>>> - $db_res = null;
>>>> -
>>>> - do_pg_connect();
>>>> -
>>>> - debug("\nSQL: $cmd\n");
>>>> - if (! ($db_res = pg_query($cmd))) {
>>>> - $out = "\nSQL QUERY: " . $cmd .
>>>> - "\nSQL ERROR: " . pg_last_error() .
>>>> - "\nSQL ERROR: " . pg_result_error();
>>>> -
>>>> - # Some errors are unsurprising, allow for silence in
>>>> - # such cases
>>>> - if (! $silent) {
>>>> - mtt_error($out);
>>>> - mtt_send_mail($out);
>>>> - }
>>>> - }
>>>> - debug("\nDatabase rows affected: " . pg_affected_rows($db_res)
>>>> . "\n");
>>>> -
>>>> - return $db_res;
>>>> -}
>>>> -
>>>> -# Fetch scalar value
>>>> -function select_scalar($cmd) {
>>>> -
>>>> - $set = array();
>>>> - $set = simple_select($cmd);
>>>> - return array_shift($set);
>>>> -}
>>>> -
>>>> -# Fetch 1D array
>>>> -function simple_select($cmd) {
>>>> -
>>>> - do_pg_connect();
>>>> -
>>>> - $rows = null;
>>>> -
>>>> - debug("\nSQL: $cmd\n");
>>>> - if (! ($result = pg_query($cmd))) {
>>>> - $out = "\nSQL QUERY: " . $cmd .
>>>> - "\nSQL ERROR: " . pg_last_error() .
>>>> - "\nSQL ERROR: " . pg_result_error();
>>>> - mtt_error($out);
>>>> - mtt_send_mail($out);
>>>> - }
>>>> - $max = pg_num_rows($result);
>>>> - for ($i = 0; $i < $max; ++$i) {
>>>> - $row = pg_fetch_array($result, $i, PGSQL_NUM);
>>>> - $rows[] = $row[0];
>>>> - }
>>>> - return $rows;
>>>> -}
>>>> -
>>>> # Fetch an associative hash (column name => value)
>>>> function associative_select($cmd) {
>>>>
>>>> @@ -1722,21 +1584,6 @@ function associative_select($cmd) {
>>>> return pg_fetch_array($result);
>>>> }
>>>>
>>>> -# Fetch 2D array
>>>> -function select($cmd) {
>>>> - do_pg_connect();
>>>> -
>>>> - debug("\nSQL: $cmd\n");
>>>> - if (! ($result = pg_query($cmd))) {
>>>> - $out = "\nSQL QUERY: " . $cmd .
>>>> - "\nSQL ERROR: " . pg_last_error() .
>>>> - "\nSQL ERROR: " . pg_result_error();
>>>> - mtt_error($out);
>>>> - mtt_send_mail($out);
>>>> - }
>>>> - return pg_fetch_all($result);
>>>> -}
>>>> -
>>>> ######################################################################
>>>>
>>>> # Function for reporting errors back to the client
>>>
>>>
>
>