Subject: Re: [MTT users] MTT server side problem
From: Ethan Mallove (ethan.mallove_at_[hidden])
Date: 2008-05-07 11:42:33
On Wed, May/07/2008 06:04:07PM, Pavel Shamis (Pasha) wrote:
> Hi Josh.
> >
> > Looking at the patch I'm a little bit conserned. The
> > "get_table_fields()" is, as you mentioned, no longer used so should be
> > removed. However the other functions are critical to the submission
> > script particularly 'do_pg_connect' which opens the connection to the
> > backend database.
> All the functions are implemented in $topdir/database.inc file. And the
> "database.inc" implementation is better because it use password and
> username from config.ini. The original implementation from submit/index use
> hardcoded values defined in the file.
> >
> > Are you using the current development trunk (mtt/trunk) or the stable
> > release branch (mtt/branches/ompi-core-testers)?
> trunk
> >
> > Can you send us the error messages that you were receiving?
>
> 1. On client side I see ""*** WARNING: MTTDatabase client did not get a
> serial"
> As result of the error some of MTT results is not visible via the web
> reporter
>
> 2. On server side I found follow error message:
> [client 10.4.3.214] PHP Fatal error: Allowed memory size of 33554432
> bytes exhausted (tried to allocate 23592960
> bytes) in /.autodirect/swgwork/MTT/mtt/submit/index.php(79) : eval()'d
> code on line 77515
> [Mon May 05 19:26:05 2008] [notice] caught SIGTERM, shutting down
> [Mon May 05 19:30:54 2008] [notice] suEXEC mechanism enabled (wrapper:
> /usr/sbin/suexec)
> [Mon May 05 19:30:54 2008] [notice] Digest: generating secret for digest
> authentication ...
> [Mon May 05 19:30:54 2008] [notice] Digest: done
> [Mon May 05 19:30:54 2008] [notice] LDAP: Built with OpenLDAP LDAP SDK
> [Mon May 05 19:30:54 2008] [notice] LDAP: SSL support unavailable
>
> My memory limit in php.ini file was set on 256MB !
Looks like PHP is actually using a 32MB limit ("Allowed
memory size of 33554432 ..."). Does a (Apache?) daemon need
to be restarted for the php.ini file to take effect? To
check your settings, this little PHP script will print an
HTML page of all the active system settings (search on
"memory_limit").
<?php
phpinfo();
?>
-Ethan
>
> Regards,
> Pasha
>
> >
> > Cheers,
> > Josh
> >
> > On May 7, 2008, at 4:49 AM, Pavel Shamis (Pasha) wrote:
> >
> >> Hi,
> >> I upgraded the server side (the mtt is still running , so don't know
> >> if the problem was resolved)
> >> During upgrade I had some problem with the submit/index.php script,
> >> it had some duplicated functions and some of them were broken.
> >> Please review the attached patch.
> >>
> >> Pasha
> >>
> >> Ethan Mallove wrote:
> >>> On Tue, May/06/2008 06:29:33PM, Pavel Shamis (Pasha) wrote:
> >>>
> >>>>> I'm not sure which cron jobs you're referring to. Do you
> >>>>> mean these?
> >>>>>
> >>>>> https://svn.open-mpi.org/trac/mtt/browser/trunk/server/php/cron
> >>>>>
> >>>> I talked about this one:
> >>>> https://svn.open-mpi.org/trac/mtt/wiki/ServerMaintenance
> >>>>
> >>>
> >>> I'm guessing you would only be concerned with the below
> >>> periodic-maintenance.pl script, which just runs
> >>> ANALYZE/VACUUM queries. I think you can start that up
> >>> whenever you want (and it should optimize the Reporter).
> >>>
> >>> https://svn.open-mpi.org/trac/mtt/browser/trunk/server/sql/cron/periodic-maintenance.pl
> >>>
> >>>
> >>> -Ethan
> >>>
> >>>
> >>>
> >>>>> The only thing there are the regular
> >>>>> mtt-results_at_[hidden] email alerts and some out-of-date
> >>>>> DB monitoring junk. You can ignore that stuff.
> >>>>>
> >>>>> Josh, are there some nightly (DB
> >>>>> pruning/cleaning/vacuuming?) cron jobs that Pasha should be
> >>>>> running?
> >>>>>
> >>>>> -Ethan
> >>>>>
> >>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>> Ethan Mallove wrote:
> >>>>>>
> >>>>>>> Hi Pasha,
> >>>>>>>
> >>>>>>> I thought this issue was solved in r1119 (see below). Do you
> >>>>>>> have the latest mtt/server scripts?
> >>>>>>>
> >>>>>>> https://svn.open-mpi.org/trac/mtt/changeset/1119/trunk/server/php/submit
> >>>>>>>
> >>>>>>>
> >>>>>>> -Ethan
> >>>>>>>
> >>>>>>> On Tue, May/06/2008 03:26:43PM, Pavel Shamis (Pasha) wrote:
> >>>>>>>
> >>>>>>>> About the issue:
> >>>>>>>> 1. On client side I see ""*** WARNING: MTTDatabase client did
> >>>>>>>> not get a serial"
> >>>>>>>> As result of the error some of MTT results is not visible via
> >>>>>>>> the web reporter
> >>>>>>>> 2. On server side I found follow error message:
> >>>>>>>> [client 10.4.3.214] PHP Fatal error: Allowed memory size of
> >>>>>>>> 33554432 bytes exhausted (tried to allocate 23592960
> >>>>>>>> bytes) in /.autodirect/swgwork/MTT/mtt/submit/index.php(79) :
> >>>>>>>> eval()'d code on line 77515
> >>>>>>>> [Mon May 05 19:26:05 2008] [notice] caught SIGTERM, shutting down
> >>>>>>>> [Mon May 05 19:30:54 2008] [notice] suEXEC mechanism enabled
> >>>>>>>> (wrapper: /usr/sbin/suexec)
> >>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: generating secret
> >>>>>>>> for digest authentication ...
> >>>>>>>> [Mon May 05 19:30:54 2008] [notice] Digest: done
> >>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: Built with OpenLDAP
> >>>>>>>> LDAP SDK
> >>>>>>>> [Mon May 05 19:30:54 2008] [notice] LDAP: SSL support unavailable
> >>>>>>>> My memory limit in php.ini file was set on 256MB !
> >>>>>>>>
> >>>>>>>> Any ideas ?
> >>>>>>>>
> >>>>>>>> Thanks.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Pavel Shamis (Pasha)
> >>>>>>>> Mellanox Technologies
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> mtt-users mailing list
> >>>>>>>> mtt-users_at_[hidden]
> >>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >>>>>>>>
> >>>>>>>
> >>>>>> --
> >>>>>> Pavel Shamis (Pasha)
> >>>>>> Mellanox Technologies
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> mtt-users mailing list
> >>>>>> mtt-users_at_[hidden]
> >>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >>>>>>
> >>>>>
> >>>> --
> >>>> Pavel Shamis (Pasha)
> >>>> Mellanox Technologies
> >>>>
> >>>> _______________________________________________
> >>>> mtt-users mailing list
> >>>> mtt-users_at_[hidden]
> >>>> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users
> >>>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Pavel Shamis (Pasha)
> >> Mellanox Technologies
> >>
> >> Index: submit/index.php
> >> ===================================================================
> >> --- submit/index.php (revision 1200)
> >> +++ submit/index.php (working copy)
> >> @@ -1,6 +1,7 @@
> >> <?php
> >> # Copyright (c) 2006 Cisco Systems, Inc. All rights reserved.
> >> # Copyright (c) 2006-2007 Sun Microsystems, Inc. All rights reserved.
> >> +# Copyright (c) 2008 Mellanox Technologies. All rights reserved.
> >>
> >> #
> >> #
> >> @@ -24,8 +25,7 @@ if (file_exists("$topdir/config.inc")) {
> >> ini_set("memory_limit", "32M");
> >>
> >> $topdir = '..';
> >> -$ompi_home = '/l/osl/www/doc/www.open-mpi.org';
> >> -include_once("$ompi_home/dbpassword.inc");
> >> +include_once("$topdir/database.inc");
> >> include_once("$topdir/reporter.inc");
> >>
> >>
> >> @@ -1465,60 +1465,6 @@ function get_table_indexes($table_name,
> >> return simple_select($sql_cmd);
> >> }
> >>
> >> -# Function used to determine which _POST fields
> >> -# to INSERT. Prevent non-existent fields from being
> >> -# INSERTed
> >> -function get_table_fields($table_name) {
> >> -
> >> - global $dbname;
> >> - global $id;
> >> -
> >> - # These indexes are special in that they link phases
> >> - # together and hence, can and do show up in _POST
> >> - if ($table_name == "test_build")
> >> - $special_indexes = array("mpi_install$id");
> >> - elseif ($table_name == "test_run")
> >> - $special_indexes = array("test_build$id");
> >> -
> >> - # Crude way to tell whether a field is an index
> >> - $is_not_index_clause =
> >> - "\n\t (table_name = '$table_name' AND NOT " .
> >> - "\n\t (data_type = 'integer' AND " .
> >> - "\n\t column_name ~ '_id$' AND " .
> >> - "\n\t table_catalog = '$dbname'))";
> >> -
> >> - $is_special_index_clause =
> >> - "\n\t (table_name = '$table_name' AND " .
> >> - "\n\t (column_name = '$special_indexes[0]'))";
> >> -
> >> - $is_index_columns = array(
> >> - "column_name",
> >> - "data_type",
> >> - "column_default");
> >> -
> >> - $sql_cmd = "\n SELECT " . join(",",$is_index_columns) .
> >> - "\n\t FROM information_schema.columns WHERE " .
> >> - "\n\t " .
> >> - $is_not_index_clause . " OR " .
> >> - $is_special_index_clause . ';';
> >> -
> >> - do_pg_connect();
> >> -
> >> - # This table will be easier to manage if it's
> >> - # keyed by column, instead of index
> >> - $tmp = array();
> >> - $arr = array();
> >> - $arr = select($sql_cmd);
> >> -
> >> - foreach ($is_index_columns as $col) {
> >> - $tmp[$col] = array();
> >> - for ($i = 0; $i < sizeof($arr); $i++) {
> >> - $tmp[$col][] = $arr[$i][$col];
> >> - }
> >> - }
> >> - return $tmp;
> >> -}
> >> -
> >> #
> >> # Useful display of all the parameters posted.
> >> # Warning: this function could explode the memory footprint causing the
> >> @@ -1622,90 +1568,6 @@ function is_null_($var) {
> >>
> >> ######################################################################
> >>
> >> -function do_pg_connect() {
> >> -
> >> - global $dbname;
> >> - global $user;
> >> - global $pass;
> >> - global $pgsql_conn;
> >> - static $connected = false;
> >> -
> >> - if (!$connected) {
> >> - $pgsql_conn =
> >> - pg_connect("host=localhost port=5432 dbname=$dbname
> >> user=$user password=$pass");
> >> -
> >> - # Exit if we cannot connect
> >> - if (!$pgsql_conn) {
> >> - mtt_abort("\nCould not connect to the $dbname database; " .
> >> - "submit this run later.");
> >> - }
> >> - else {
> >> - $connected = true;
> >> -
> >> - # Serialize all transactions by default - Safer that way.
> >> - do_pg_query("SET SESSION CHARACTERISTICS AS TRANSACTION
> >> ISOLATION LEVEL SERIALIZABLE", false);
> >> - }
> >> -
> >> - # pg_set_error_verbosity($pgsql_conn, PGSQL_ERRORS_VERBOSE);
> >> # PHP 5 needed
> >> - # pg_trace($_ENV['HOME'] . "/pgsql.trace", 'w', $pgsql_conn);
> >> - # debug("\npg_options: " .
> >> var_export(pg_options($pgsql_conn)));
> >> - }
> >> -}
> >> -
> >> -function do_pg_query($cmd, $silent) {
> >> - $db_res = null;
> >> -
> >> - do_pg_connect();
> >> -
> >> - debug("\nSQL: $cmd\n");
> >> - if (! ($db_res = pg_query($cmd))) {
> >> - $out = "\nSQL QUERY: " . $cmd .
> >> - "\nSQL ERROR: " . pg_last_error() .
> >> - "\nSQL ERROR: " . pg_result_error();
> >> -
> >> - # Some errors are unsurprising, allow for silence in
> >> - # such cases
> >> - if (! $silent) {
> >> - mtt_error($out);
> >> - mtt_send_mail($out);
> >> - }
> >> - }
> >> - debug("\nDatabase rows affected: " . pg_affected_rows($db_res) .
> >> "\n");
> >> -
> >> - return $db_res;
> >> -}
> >> -
> >> -# Fetch scalar value
> >> -function select_scalar($cmd) {
> >> -
> >> - $set = array();
> >> - $set = simple_select($cmd);
> >> - return array_shift($set);
> >> -}
> >> -
> >> -# Fetch 1D array
> >> -function simple_select($cmd) {
> >> -
> >> - do_pg_connect();
> >> -
> >> - $rows = null;
> >> -
> >> - debug("\nSQL: $cmd\n");
> >> - if (! ($result = pg_query($cmd))) {
> >> - $out = "\nSQL QUERY: " . $cmd .
> >> - "\nSQL ERROR: " . pg_last_error() .
> >> - "\nSQL ERROR: " . pg_result_error();
> >> - mtt_error($out);
> >> - mtt_send_mail($out);
> >> - }
> >> - $max = pg_num_rows($result);
> >> - for ($i = 0; $i < $max; ++$i) {
> >> - $row = pg_fetch_array($result, $i, PGSQL_NUM);
> >> - $rows[] = $row[0];
> >> - }
> >> - return $rows;
> >> -}
> >> -
> >> # Fetch an associative hash (column name => value)
> >> function associative_select($cmd) {
> >>
> >> @@ -1722,21 +1584,6 @@ function associative_select($cmd) {
> >> return pg_fetch_array($result);
> >> }
> >>
> >> -# Fetch 2D array
> >> -function select($cmd) {
> >> - do_pg_connect();
> >> -
> >> - debug("\nSQL: $cmd\n");
> >> - if (! ($result = pg_query($cmd))) {
> >> - $out = "\nSQL QUERY: " . $cmd .
> >> - "\nSQL ERROR: " . pg_last_error() .
> >> - "\nSQL ERROR: " . pg_result_error();
> >> - mtt_error($out);
> >> - mtt_send_mail($out);
> >> - }
> >> - return pg_fetch_all($result);
> >> -}
> >> -
> >> ######################################################################
> >>
> >> # Function for reporting errors back to the client
> >
> >
>
>
> --
> Pavel Shamis (Pasha)
> Mellanox Technologies
>
> _______________________________________________
> mtt-users mailing list
> mtt-users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users