Connection to a proxy with authentication

[elnuno]

I use PTS on Linux embedded devices, connected to the external network via a company proxy.

I'm on Phoronix Test Suite version v6.8.0.
To automate the proxy connection I have added the following tags to phoronix-test-suite.xml

    <Networking>
      <NoInternetCommunication>FALSE</NoInternetCommunication>
      <NoNetworkCommunication>FALSE</NoNetworkCommunication>
      <Timeout>20</Timeout>
      <ProxyAddress>proxy_address</ProxyAddress>
      <ProxyPort>8080</ProxyPort>
      <ProxyUser>login</ProxyUser>
      <ProxyPassword>pwd</ProxyPassword>
    </Networking>

I also add to modify a bit the PHP code to take these parameters into account. Default version did not allow me to enter login and password. So my PTS version is a patched one.

I wonder whether this is the right way to unlock the authentication stage for a proxy connection:
- is this corrected/implemented in a more recent PTS version?
- Is there a more clever way to support authentication with the current version?

Thanks in advance

[Michael]

I use PTS on Linux embedded devices, connected to the external network via a company proxy.

I'm on Phoronix Test Suite version v6.8.0.
To automate the proxy connection I have added the following tags to phoronix-test-suite.xml

<Networking>
<NoInternetCommunication>FALSE</NoInternetCommunication>
<NoNetworkCommunication>FALSE</NoNetworkCommunication>
<Timeout>20</Timeout>
<ProxyAddress>proxy_address</ProxyAddress>
<ProxyPort>8080</ProxyPort>
<ProxyUser>login</ProxyUser>
<ProxyPassword>pwd</ProxyPassword>
</Networking>

I also add to modify a bit the PHP code to take these parameters into account. Default version did not allow me to enter login and password. So my PTS version is a patched one.

I wonder whether this is the right way to unlock the authentication stage for a proxy connection:
- is this corrected/implemented in a more recent PTS version?
- Is there a more clever way to support authentication with the current version?

Thanks in advance

Hi, are you able to post your patch for this? I can look at it and try to clean it up and merge it. Unfortunately I don't have any proxy authentication systems to test with so basically rely upon testing and feedback from others.

[elnuno]

Sorry, i'm not allowed to attach the file, here is the whole content.

From ce009b41ab6932aba98273b03f2a098b21280877 Mon Sep 17 00:00:00 2001
From: 
Date: Mon, 20 Feb 2017 14:12:46 +0100
Subject: [PATCH 1/1] Add proxy authentification

Signed-off-by: 
---
 pts-core/commands/network_setup.php    |  8 ++++++--
 pts-core/objects/client/pts_config.php |  2 ++
 pts-core/objects/pts_network.php       | 24 +++++++++++++++++++-----
 pts-core/objects/pts_user_io.php       | 25 ++++++++++++++++++++++++-
 4 files changed, 51 insertions(+), 8 deletions(-)

diff --git a/pts-core/commands/network_setup.php b/pts-core/commands/network_setup.php
index 534fc60..532bbf1 100644
--- a/pts-core/commands/network_setup.php
+++ b/pts-core/commands/network_setup.php
@@ -36,15 +36,19 @@ class network_setup implements pts_option_interface

         $proxy_address = pts_user_io::prompt_user_input('Enter IP address / server name of proxy');
         $proxy_port = pts_user_io::prompt_user_input('Enter TCP port for proxy server');
+        $proxy_user = pts_user_io::prompt_user_input('Enter User name for proxy ');
+        $proxy_password = pts_user_io::strToHex(pts_user_io::prompt_user_input('Enter password for proxy', false, true));

         echo PHP_EOL . 'Testing Proxy Server (' . $proxy_address . ':' . $proxy_port . ')' . PHP_EOL;

-        if(pts_network::http_get_contents('http://www.phoronix-test-suite.com/PTS', $proxy_address, $proxy_port) == 'PTS')
+        if(pts_network::http_get_contents('http://www.phoronix-test-suite.com/PTS', $proxy_address, $proxy_port, $proxy_user, $proxy_password) == 'PTS')
         {
             echo PHP_EOL . 'Proxy Setup Completed; Storing Network Settings.' . PHP_EOL;
             pts_config::user_config_generate(array(
                 'PhoronixTestSuite/Options/Networking/ProxyAddress' => $proxy_address,
-                'PhoronixTestSuite/Options/Networking/ProxyPort' => $proxy_port
+                'PhoronixTestSuite/Options/Networking/ProxyPort' => $proxy_port,
+                'PhoronixTestSuite/Options/Networking/ProxyUser' => $proxy_user,
+                'PhoronixTestSuite/Options/Networking/ProxyPassword' => $proxy_password
                 ));
         }
         else
diff --git a/pts-core/objects/client/pts_config.php b/pts-core/objects/client/pts_config.php
index 6b6500d..bd90817 100644
--- a/pts-core/objects/client/pts_config.php
+++ b/pts-core/objects/client/pts_config.php
@@ -138,6 +138,8 @@ class pts_config
         $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/Timeout', $read_config);
         $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyAddress', $read_config);
         $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyPort', $read_config);
+        $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyUser', $read_config);
+        $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyPassword', $read_config);

         $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Server/RemoteAccessPort', $read_config);
         $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Server/Password', $read_config);
diff --git a/pts-core/objects/pts_network.php b/pts-core/objects/pts_network.php
index cfcf93e..716480f 100644
--- a/pts-core/objects/pts_network.php
+++ b/pts-core/objects/pts_network.php
@@ -39,14 +39,14 @@ class pts_network
     {
         return self::$disable_network_support == false;
     }
-    public static function http_get_contents($url, $override_proxy = false, $override_proxy_port = false, $http_timeout = -1)
+    public static function http_get_contents($url, $override_proxy = false, $override_proxy_port = false, $proxy_user =false, $proxy_password=false, $http_timeout = -1)
     {
         if(!pts_network::network_support_available())
         {
             return false;
         }

-        $stream_context = pts_network::stream_context_create(null, $override_proxy, $override_proxy_port, $http_timeout);
+        $stream_context = pts_network::stream_context_create(null, $override_proxy, $override_proxy_port, $proxy_user, $proxy_password, $http_timeout);
         $contents = pts_file_io::file_get_contents($url, 0, $stream_context);

         return $contents;
@@ -162,6 +162,7 @@ class pts_network
         if(self::$network_proxy)
         {
             curl_setopt($cr, CURLOPT_PROXY, self::$network_proxy['proxy']);
+            curl_setopt($ch, CURLOPT_USERPWD, self::$network_proxy['user'] . ":" . self::$network_proxy['password']);
         }

         curl_exec($cr);
@@ -200,7 +201,7 @@ class pts_network

         return false;
     }
-    public static function stream_context_create($parameters = null, $proxy_address = false, $proxy_port = false, $http_timeout = -1)
+    public static function stream_context_create($parameters = null, $proxy_address  = false, $proxy_port = false, $proxy_user = false, $proxy_password = false, $http_timeout = -1)
     {
         if(!is_array($parameters))
         {
@@ -211,6 +212,8 @@ class pts_network
         {
             $proxy_address = self::$network_proxy['address'];
             $proxy_port = self::$network_proxy['port'];
+            $proxy_user = self::$network_proxy['user'];
+            $proxy_password = self::$network_proxy['password'];
         }

         if($proxy_address != false && $proxy_port != false && is_numeric($proxy_port) && $proxy_port > 1)
@@ -229,8 +232,16 @@ class pts_network
         }

         $parameters['http']['user_agent'] = pts_core::codename(true);
-        $parameters['http']['header'] = "Content-Type: application/x-www-form-urlencoded\r\n";
-
+        if ($proxy_user != false)
+        {
+            $password=pts_user_io::hexToStr($proxy_password);
+            $auth = base64_encode("$proxy_user:$password");
+            $parameters['http']['header'] = "Proxy-Authorization: Basic $auth";
+        }
+        else
+        {
+            $parameters['http']['header'] = "Content-Type: application/x-www-form-urlencoded\r\n";
+        }
         $stream_context = stream_context_create($parameters);

         return $stream_context;
@@ -287,6 +298,9 @@ class pts_network
             self::$network_proxy['proxy'] = $proxy_address . ':' . $proxy_port;
             self::$network_proxy['address'] = $proxy_address;
             self::$network_proxy['port'] = $proxy_port;
+            self::$network_proxy['user'] = pts_config::read_user_config('PhoronixTestSuite/Options/Networking/ProxyUser', false);
+            self::$network_proxy['password'] = pts_config::read_user_config('PhoronixTestSuite/Options/Networking/ProxyPassword', false);
+
         }
         else if(($env_proxy = getenv('http_proxy')) != false && count($env_proxy = pts_strings::colon_explode($env_proxy)) == 2)
         {
diff --git a/pts-core/objects/pts_user_io.php b/pts-core/objects/pts_user_io.php
index db49051..664290b 100644
--- a/pts-core/objects/pts_user_io.php
+++ b/pts-core/objects/pts_user_io.php
@@ -22,16 +22,39 @@

 class pts_user_io
 {
+    public static function strToHex($string)
+    {
+        $hex = '';
+        for ($i=0; $i<strlen($string); $i++){
+            $ord = ord($string[$i]);
+            $hexCode = dechex($ord);
+            $hex .= substr('0'.$hexCode, -2);
+        }
+        return strToUpper($hex);
+    }
+
+    public static function hexToStr($hex){
+        $string='';
+        for ($i=0; $i < strlen($hex)-1; $i+=2){
+            $string .= chr(hexdec($hex[$i].$hex[$i+1]));
+        }
+        return $string;
+    }
+
     public static function read_user_input()
     {
         return trim(fgets(STDIN));
     }
-    public static function prompt_user_input($question, $allow_null = false)
+    public static function prompt_user_input($question, $allow_null = false, $password = false)
     {
         do
         {
             echo PHP_EOL . $question . ': ';
+            if ($password)
+                system('stty -echo');
             $answer = pts_user_io::read_user_input();
+            if ($password)
+                system('stty echo');
         }
         while(!$allow_null && empty($answer));

-- 
2.7.4

[Michael]

Sorry, i'm not allowed to attach the file, here is the whole content.

From ce009b41ab6932aba98273b03f2a098b21280877 Mon Sep 17 00:00:00 2001
From:
Date: Mon, 20 Feb 2017 14:12:46 +0100
Subject: [PATCH 1/1] Add proxy authentification

Signed-off-by:
---
pts-core/commands/network_setup.php | 8 ++++++--
pts-core/objects/client/pts_config.php | 2 ++
pts-core/objects/pts_network.php | 24 +++++++++++++++++++-----
pts-core/objects/pts_user_io.php | 25 ++++++++++++++++++++++++-
4 files changed, 51 insertions(+), 8 deletions(-)

diff --git a/pts-core/commands/network_setup.php b/pts-core/commands/network_setup.php
index 534fc60..532bbf1 100644
--- a/pts-core/commands/network_setup.php
+++ b/pts-core/commands/network_setup.php
@@ -36,15 +36,19 @@ class network_setup implements pts_option_interface

$proxy_address = pts_user_io::prompt_user_input('Enter IP address / server name of proxy');
$proxy_port = pts_user_io::prompt_user_input('Enter TCP port for proxy server');
+ $proxy_user = pts_user_io::prompt_user_input('Enter User name for proxy ');
+ $proxy_password = pts_user_io::strToHex(pts_user_io::prompt_user_input('Enter password for proxy', false, true));

echo PHP_EOL . 'Testing Proxy Server (' . $proxy_address . ':' . $proxy_port . ')' . PHP_EOL;

- if(pts_network::http_get_contents('http://www.phoronix-test-suite.com/PTS', $proxy_address, $proxy_port) == 'PTS')
+ if(pts_network::http_get_contents('http://www.phoronix-test-suite.com/PTS', $proxy_address, $proxy_port, $proxy_user, $proxy_password) == 'PTS')
{
echo PHP_EOL . 'Proxy Setup Completed; Storing Network Settings.' . PHP_EOL;
pts_config::user_config_generate(array(
'PhoronixTestSuite/Options/Networking/ProxyAddress' => $proxy_address,
- 'PhoronixTestSuite/Options/Networking/ProxyPort' => $proxy_port
+ 'PhoronixTestSuite/Options/Networking/ProxyPort' => $proxy_port,
+ 'PhoronixTestSuite/Options/Networking/ProxyUser' => $proxy_user,
+ 'PhoronixTestSuite/Options/Networking/ProxyPassword' => $proxy_password
));
}
else
diff --git a/pts-core/objects/client/pts_config.php b/pts-core/objects/client/pts_config.php
index 6b6500d..bd90817 100644
--- a/pts-core/objects/client/pts_config.php
+++ b/pts-core/objects/client/pts_config.php
@@ -138,6 +138,8 @@ class pts_config
$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/Timeout', $read_config);
$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyAddress', $read_config);
$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyPort', $read_config);
+ $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyUser', $read_config);
+ $config->addXmlNodeFromReader('PhoronixTestSuite/Options/Networking/ProxyPassword', $read_config);

$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Server/RemoteAccessPort', $read_config);
$config->addXmlNodeFromReader('PhoronixTestSuite/Options/Server/Password', $read_config);
diff --git a/pts-core/objects/pts_network.php b/pts-core/objects/pts_network.php
index cfcf93e..716480f 100644
--- a/pts-core/objects/pts_network.php
+++ b/pts-core/objects/pts_network.php
@@ -39,14 +39,14 @@ class pts_network
{
return self::$disable_network_support == false;
}
- public static function http_get_contents($url, $override_proxy = false, $override_proxy_port = false, $http_timeout = -1)
+ public static function http_get_contents($url, $override_proxy = false, $override_proxy_port = false, $proxy_user =false, $proxy_password=false, $http_timeout = -1)
{
if(!pts_network::network_support_available())
{
return false;
}

- $stream_context = pts_network::stream_context_create(null, $override_proxy, $override_proxy_port, $http_timeout);
+ $stream_context = pts_network::stream_context_create(null, $override_proxy, $override_proxy_port, $proxy_user, $proxy_password, $http_timeout);
$contents = pts_file_io::file_get_contents($url, 0, $stream_context);

return $contents;
@@ -162,6 +162,7 @@ class pts_network
if(self::$network_proxy)
{
curl_setopt($cr, CURLOPT_PROXY, self::$network_proxy['proxy']);
+ curl_setopt($ch, CURLOPT_USERPWD, self::$network_proxy['user'] . ":" . self::$network_proxy['password']);
}

curl_exec($cr);
@@ -200,7 +201,7 @@ class pts_network

return false;
}
- public static function stream_context_create($parameters = null, $proxy_address = false, $proxy_port = false, $http_timeout = -1)
+ public static function stream_context_create($parameters = null, $proxy_address = false, $proxy_port = false, $proxy_user = false, $proxy_password = false, $http_timeout = -1)
{
if(!is_array($parameters))
{
@@ -211,6 +212,8 @@ class pts_network
{
$proxy_address = self::$network_proxy['address'];
$proxy_port = self::$network_proxy['port'];
+ $proxy_user = self::$network_proxy['user'];
+ $proxy_password = self::$network_proxy['password'];
}

if($proxy_address != false && $proxy_port != false && is_numeric($proxy_port) && $proxy_port > 1)
@@ -229,8 +232,16 @@ class pts_network
}

$parameters['http']['user_agent'] = pts_core::codename(true);
- $parameters['http']['header'] = "Content-Type: application/x-www-form-urlencoded\r\n";
-
+ if ($proxy_user != false)
+ {
+ $password=pts_user_io::hexToStr($proxy_password);
+ $auth = base64_encode("$proxy_user:$password");
+ $parameters['http']['header'] = "Proxy-Authorization: Basic $auth";
+ }
+ else
+ {
+ $parameters['http']['header'] = "Content-Type: application/x-www-form-urlencoded\r\n";
+ }
$stream_context = stream_context_create($parameters);

return $stream_context;
@@ -287,6 +298,9 @@ class pts_network
self::$network_proxy['proxy'] = $proxy_address . ':' . $proxy_port;
self::$network_proxy['address'] = $proxy_address;
self::$network_proxy['port'] = $proxy_port;
+ self::$network_proxy['user'] = pts_config::read_user_config('PhoronixTestSuite/Options/Networking/ProxyUser', false);
+ self::$network_proxy['password'] = pts_config::read_user_config('PhoronixTestSuite/Options/Networking/ProxyPassword', false);
+
}
else if(($env_proxy = getenv('http_proxy')) != false && count($env_proxy = pts_strings::colon_explode($env_proxy)) == 2)
{
diff --git a/pts-core/objects/pts_user_io.php b/pts-core/objects/pts_user_io.php
index db49051..664290b 100644
--- a/pts-core/objects/pts_user_io.php
+++ b/pts-core/objects/pts_user_io.php
@@ -22,16 +22,39 @@

class pts_user_io
{
+ public static function strToHex($string)
+ {
+ $hex = '';
+ for ($i=0; $i<strlen($string); $i++){
+ $ord = ord($string[$i]);
+ $hexCode = dechex($ord);
+ $hex .= substr('0'.$hexCode, -2);
+ }
+ return strToUpper($hex);
+ }
+
+ public static function hexToStr($hex){
+ $string='';
+ for ($i=0; $i < strlen($hex)-1; $i+=2){
+ $string .= chr(hexdec($hex[$i].$hex[$i+1]));
+ }
+ return $string;
+ }
+
public static function read_user_input()
{
return trim(fgets(STDIN));
}
- public static function prompt_user_input($question, $allow_null = false)
+ public static function prompt_user_input($question, $allow_null = false, $password = false)
{
do
{
echo PHP_EOL . $question . ': ';
+ if ($password)
+ system('stty -echo');
$answer = pts_user_io::read_user_input();
+ if ($password)
+ system('stty echo');
}
while(!$allow_null && empty($answer));

--
2.7.4

From a quick look over, it appears to be properly hooked up. Is it working fine for you on your end? Will review it more thoroughly later today.

[elnuno]

It half works. I can't download a few archives, such as:

 Download Failed: ftp://ftp.pureftpd.org/pub/pure-ftpd/misc/phpbench/phpbench-0.8.1.tar.gz
            Attempting to download from alternate mirror.
        Downloading: phpbench-0.8.1.tar.gz [0.01MB]
            Download Failed: http://download.pureftpd.org/pub/phpbench/phpbench-0.8.1.tar.gz
            Downloading of needed test files failed.

I don't think it can come from my network configuration, as I can download using wget after setting a ftp_proxy env variable.
It seems to happen only on repositories hosted by sourceforge or pureftpd. I'm not an expert all, it may be a timeout or redirection issue.

[Michael]

It half works. I can't download a few archives, such as:

Download Failed: ftp://ftp.pureftpd.org/pub/pure-ftpd/misc/phpbench/phpbench-0.8.1.tar.gz
Attempting to download from alternate mirror.
Downloading: phpbench-0.8.1.tar.gz [0.01MB]
Download Failed: http://download.pureftpd.org/pub/phpbench/phpbench-0.8.1.tar.gz
Downloading of needed test files failed.

I don't think it can come from my network configuration, as I can download using wget after setting a ftp_proxy env variable.
It seems to happen only on repositories hosted by sourceforge or pureftpd. I'm not an expert all, it may be a timeout or redirection issue.

A cleaned up version of the support can be found in Git now @ https://github.com/phoronix-test-sui...bad3601b939ebd

Thanks, if you're able to test it in Git that would be great.

[elnuno]

I will test this, probably tomorrow morning CET. Thanks Michael

[elnuno]

Here is the result of my test. I overwrote all the files with your cleaned content.

I met an error with pts_client::cli_just_bold(), undefined in my 6.8 version. After suppressing the calls to this method, I managed to setup a proxy with authentication.
I could then benchmark pts/tscp without any issue.

Though, I still meet a problem with pts/system-decompress-bzip2. Download fails. At that stage I can't be sure whether this is a pure PTS issue or an issue in the network config of my target.