Subject: Re: [MTT users] MTT server side problem
From: Josh Hursey (jjhursey_at_[hidden])
Date: 2008-05-07 09:48:24
Pasha,
Looking at the patch I'm a little bit conserned. The
"get_table_fields()" is, as you mentioned, no longer used so should be
removed. However the other functions are critical to the submission
script particularly 'do_pg_connect' which opens the connection to the
backend database.
Are you using the current development trunk (mtt/trunk) or the stable
release branch (mtt/branches/ompi-core-testers)?
Can you send us the error messages that you were receiving?
Cheers,
Josh
On May 7, 2008, at 4:49 AM, Pavel Shamis (Pasha) wrote:
> Hi,
> I upgraded the server side (the mtt is still running , so don't know
> if the problem was resolved)
> During upgrade I had some problem with the submit/index.php script,
> it had some duplicated functions and some of them were broken.
> Please review the attached patch.
>
> Pasha
>
> Ethan Mallove wrote:
>> On Tue, May/06/2008 06:29:33PM, Pavel Shamis (Pasha) wrote:
>>
>>>> I'm not sure which cron jobs you're referring to. Do you
>>>> mean these?
>>>>
>>>> https://svn.open-mpi.org/trac/mtt/browser/trunk/server/php/cron
>>>>
>>> I talked about this one: https://svn.open-mpi.org/trac/mtt/wiki/ServerMaintenance
>>>
>>
>> I'm guessing you would only be concerned with the below
>> periodic-maintenance.pl script, which just runs
>> ANALYZE/VACUUM queries. I think you can start that up
>> whenever you want (and it should optimize the Reporter).
>>
>> https://svn.open-mpi.org/trac/mtt/browser/trunk/server/sql/cron/periodic-maintenance.pl
>>
>> -Ethan
>>
>>
>>
>>>> The only thing there are the regular
>>>> mtt-results_at_[hidden] email alerts and some out-of-date
>>>> DB monitoring junk. You can ignore that stuff.
>>>>
>>>> Josh, are there some nightly (DB
>>>> pruning/cleaning/vacuuming?) cron jobs that Pasha should be
>>>> running?
>>>>
>>>> -Ethan
>>>>
>>>>
>>>>> Thanks.
>>>>>
>>>>> Ethan Mallove wrote:
>>>>>
>>>>>> Hi Pasha,
>>>>>>
>>>>>> I thought this issue was solved in r1119 (see below). Do you
>>>>>> have the latest mtt/server scripts?
>>>>>>
>>>>>> https://svn.open-mpi.org/trac/mtt/changeset/1119/trunk/server/php/submit
>>>>>>
>>>>>> -Ethan
>>>>>>
>>>>>> On Tue, May/06/2008 03:26:43PM, Pavel Shamis (Pasha) wrote:
>>>>>>
>>>>>>> About the issue:
>>>>>>> 1. On client side I see ""*** WARNING: MTTDatabase client did
>>>>>>> not get a serial"
>>>>>>> As result of the error some of MTT results is not visible via
>>>>>>> the web reporter
>>>>>>> 2. On server side I found follow error message:
>>>>>>> [client 10.4.3.214] PHP Fatal error: Allowed memory size of
>>>>>>> 33554432 bytes exhausted (tried to allocate 23592960
>>>>>>> bytes) in /.autodirect/swgwork/MTT/mtt/submit/index.php(79) :
>>>>>>> eval()'d code on line 77515
>>>>>>> [Mon May 05 19:26:05 2008] [notice] caught SIGTERM, shutting
>>>>>>> down
>>>>>>> [Mon May 05 19:30:54 2008] [notice] suEXEC mechanism enabled
>>>>>>> (wrapper: /usr/sbin/suexec)
>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: generating secret
>>>>>>> for digest authentication ...
>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: done
>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: Built with OpenLDAP
>>>>>>> LDAP SDK
>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: SSL support
>>>>>>> unavailable
>>>>>>> My memory limit in php.ini file was set on 256MB !
>>>>>>>
>>>>>>> Any ideas ?
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Pavel Shamis (Pasha)
>>>>>>> Mellanox Technologies
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> mtt-users mailing list
>>>>>>> mtt-users_at_[hidden]
>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>>>
>>>>>>
>>>>> --
>>>>> Pavel Shamis (Pasha)
>>>>> Mellanox Technologies
>>>>>
>>>>> _______________________________________________
>>>>> mtt-users mailing list
>>>>> mtt-users_at_[hidden]
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>>>
>>>>
>>> --
>>> Pavel Shamis (Pasha)
>>> Mellanox Technologies
>>>
>>> _______________________________________________
>>> mtt-users mailing list
>>> mtt-users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
>>>
>>
>>
>
>
> --
> Pavel Shamis (Pasha)
> Mellanox Technologies
>
> Index: submit/index.php
> ===================================================================
> --- submit/index.php (revision 1200)
> +++ submit/index.php (working copy)
> @@ -1,6 +1,7 @@
> <?php
> # Copyright (c) 2006 Cisco Systems, Inc. All rights reserved.
> # Copyright (c) 2006-2007 Sun Microsystems, Inc. All rights reserved.
> +# Copyright (c) 2008 Mellanox Technologies. All rights
> reserved.
>
> #
> #
> @@ -24,8 +25,7 @@ if (file_exists("$topdir/config.inc")) {
> ini_set("memory_limit", "32M");
>
> $topdir = '..';
> -$ompi_home = '/l/osl/www/doc/www.open-mpi.org';
> -include_once("$ompi_home/dbpassword.inc");
> +include_once("$topdir/database.inc");
> include_once("$topdir/reporter.inc");
>
>
> @@ -1465,60 +1465,6 @@ function get_table_indexes($table_name,
> return simple_select($sql_cmd);
> }
>
> -# Function used to determine which _POST fields
> -# to INSERT. Prevent non-existent fields from being
> -# INSERTed
> -function get_table_fields($table_name) {
> -
> - global $dbname;
> - global $id;
> -
> - # These indexes are special in that they link phases
> - # together and hence, can and do show up in _POST
> - if ($table_name == "test_build")
> - $special_indexes = array("mpi_install$id");
> - elseif ($table_name == "test_run")
> - $special_indexes = array("test_build$id");
> -
> - # Crude way to tell whether a field is an index
> - $is_not_index_clause =
> - "\n\t (table_name = '$table_name' AND NOT " .
> - "\n\t (data_type = 'integer' AND " .
> - "\n\t column_name ~ '_id$' AND " .
> - "\n\t table_catalog = '$dbname'))";
> -
> - $is_special_index_clause =
> - "\n\t (table_name = '$table_name' AND " .
> - "\n\t (column_name = '$special_indexes[0]'))";
> -
> - $is_index_columns = array(
> - "column_name",
> - "data_type",
> - "column_default");
> -
> - $sql_cmd = "\n SELECT " . join(",",$is_index_columns) .
> - "\n\t FROM information_schema.columns WHERE " .
> - "\n\t " .
> - $is_not_index_clause . " OR " .
> - $is_special_index_clause . ';';
> -
> - do_pg_connect();
> -
> - # This table will be easier to manage if it's
> - # keyed by column, instead of index
> - $tmp = array();
> - $arr = array();
> - $arr = select($sql_cmd);
> -
> - foreach ($is_index_columns as $col) {
> - $tmp[$col] = array();
> - for ($i = 0; $i < sizeof($arr); $i++) {
> - $tmp[$col][] = $arr[$i][$col];
> - }
> - }
> - return $tmp;
> -}
> -
> #
> # Useful display of all the parameters posted.
> # Warning: this function could explode the memory footprint causing
> the
> @@ -1622,90 +1568,6 @@ function is_null_($var) {
>
> ######################################################################
>
> -function do_pg_connect() {
> -
> - global $dbname;
> - global $user;
> - global $pass;
> - global $pgsql_conn;
> - static $connected = false;
> -
> - if (!$connected) {
> - $pgsql_conn =
> - pg_connect("host=localhost port=5432 dbname=$dbname
> user=$user password=$pass");
> -
> - # Exit if we cannot connect
> - if (!$pgsql_conn) {
> - mtt_abort("\nCould not connect to the $dbname database;
> " .
> - "submit this run later.");
> - }
> - else {
> - $connected = true;
> -
> - # Serialize all transactions by default - Safer that way.
> - do_pg_query("SET SESSION CHARACTERISTICS AS TRANSACTION
> ISOLATION LEVEL SERIALIZABLE", false);
> - }
> -
> - # pg_set_error_verbosity($pgsql_conn,
> PGSQL_ERRORS_VERBOSE); # PHP 5 needed
> - # pg_trace($_ENV['HOME'] . "/pgsql.trace", 'w', $pgsql_conn);
> - # debug("\npg_options: " .
> var_export(pg_options($pgsql_conn)));
> - }
> -}
> -
> -function do_pg_query($cmd, $silent) {
> - $db_res = null;
> -
> - do_pg_connect();
> -
> - debug("\nSQL: $cmd\n");
> - if (! ($db_res = pg_query($cmd))) {
> - $out = "\nSQL QUERY: " . $cmd .
> - "\nSQL ERROR: " . pg_last_error() .
> - "\nSQL ERROR: " . pg_result_error();
> -
> - # Some errors are unsurprising, allow for silence in
> - # such cases
> - if (! $silent) {
> - mtt_error($out);
> - mtt_send_mail($out);
> - }
> - }
> - debug("\nDatabase rows affected: " .
> pg_affected_rows($db_res) . "\n");
> -
> - return $db_res;
> -}
> -
> -# Fetch scalar value
> -function select_scalar($cmd) {
> -
> - $set = array();
> - $set = simple_select($cmd);
> - return array_shift($set);
> -}
> -
> -# Fetch 1D array
> -function simple_select($cmd) {
> -
> - do_pg_connect();
> -
> - $rows = null;
> -
> - debug("\nSQL: $cmd\n");
> - if (! ($result = pg_query($cmd))) {
> - $out = "\nSQL QUERY: " . $cmd .
> - "\nSQL ERROR: " . pg_last_error() .
> - "\nSQL ERROR: " . pg_result_error();
> - mtt_error($out);
> - mtt_send_mail($out);
> - }
> - $max = pg_num_rows($result);
> - for ($i = 0; $i < $max; ++$i) {
> - $row = pg_fetch_array($result, $i, PGSQL_NUM);
> - $rows[] = $row[0];
> - }
> - return $rows;
> -}
> -
> # Fetch an associative hash (column name => value)
> function associative_select($cmd) {
>
> @@ -1722,21 +1584,6 @@ function associative_select($cmd) {
> return pg_fetch_array($result);
> }
>
> -# Fetch 2D array
> -function select($cmd) {
> - do_pg_connect();
> -
> - debug("\nSQL: $cmd\n");
> - if (! ($result = pg_query($cmd))) {
> - $out = "\nSQL QUERY: " . $cmd .
> - "\nSQL ERROR: " . pg_last_error() .
> - "\nSQL ERROR: " . pg_result_error();
> - mtt_error($out);
> - mtt_send_mail($out);
> - }
> - return pg_fetch_all($result);
> -}
> -
> ######################################################################
>
> # Function for reporting errors back to the client