1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
|
<html>
<title>Ganymed SSH-2 for Java FAQ</title>
<body>
<a name="oben"></a>
<h1>Ganymed SSH-2 for Java FAQ</h1>
<p>
This FAQ includes information regarding topics that were discussed in e-mails between developers and users
of the Ganymed SSH-2 for Java library.
</p>
<p>
Ganymed SSH-2 for Java homepage: <a href="http://www.cleondris.ch/opensource/ssh2/">http://www.cleondris.ch/opensource/ssh2/</a><br>
Last update of FAQ: apr-16-2010.
</p>
<p>
Please report bugs, typos and any kind of suggestions to Christian Plattner (christian.plattner at cleondris.ch).
</p>
<hr>
<h2>Sections:</h2>
<p>
<ul>
<li><a href="#env">When I start program XYZ with putty (or openssh, ..., whatever) then everything works.
However, if I use "Session.execCommand", then XYZ behaves differently or does not work at all!</a></li>
<li><a href="#blocking">My program sometimes hangs when I only read output from stdout!
Or: can you explain me the story about the shared stdout/stderr window in the SSH-2 protocol?
Or: what is this "StreamGobbler" thing all about?</a></li>
<li><a href="#buffered">Why are the session's Input- and OutputStreams not buffered?</a></li>
<li><a href="#sessioncommands">Why can't I execute several commands in one single session?</a></li>
<li><a href="#sessionlimit">I cannot open more than 10 concurrent sessions (or SCP clients).</a></li>
<li><a href="#passwordauth">Password authentication fails, I get "Authentication method password not
supported by the server at this stage".</a></li>
<li><a href="#puttygen">Why does public key authentication fail with my putty key?</a></li>
<li><a href="#catmethod">I am sending data to a remote file using the "cat" method, but not all data is being written.</a></li>
<li><a href="#pumptoremote">I want to pump data into a remote file, but the amount of data to be sent
is not known at the time the transfer starts.</a></li>
<li><a href="#swingshell">Do you have an example for the usage of feature XYZ?</a></li>
<li><a href="#maven">Where is the official Maven repository?</a></li>
</ul>
</p>
<hr><a name="env"></a><h2>When I start program XYZ with putty (or openssh, ..., whatever) then everything
works. However, if I use "Session.execCommand", then XYZ behaves differently or does not work at all!</h2>
<h3>Short answer:</h3>
<p>
The most often source of problems when executing a command with <tt>Session.execCommand()</tt>
are missing/wrong set environment variables on the remote machine. Make sure that the minimum needed
environment for XYZ is the same, independentely on how the shell is being invoked.
</p>
<p>
Example quickfix for bash users:
</p>
<p>
<ol>
<li>Define all your settings in the file <tt><b>~/.bashrc</b></tt></li>
<li>Make sure that the file <tt><b>~/.bash_profile</b></tt> only contains the line <tt><b>source
~/.bashrc</b></tt>.</li>
<li>Before executing <tt>Session.execCommand()</tt>, do NOT aquire any type of pseudo terminal in the
session. Be prepared to consume stdout and stderr data.</li>
</ol>
</p>
<p>
<b>Note:</b> If you really want to mimic the behavior of putty, then don't use Session.execCommand(),
instead aquire a pty (pseudo terminal) and then start a shell (use <tt>Session.requestPTY()</tt> and
<tt>Session.startShell()</tt>). You then have to communicate with the shell process at the other end
through stdin and stdout. However, you also have to implement terminal logic (e.g., escape sequence
handling (unless you use a "dumb" pty), "expect-send" logic (output parsing, shell prompt detection), etc.).
</p>
<h3>Long answer:</h3>
<p>
If you login by using putty, then putty will normally request a "xterm" pty and your assigned shell
(e.g., bash) will be started (a so called "interactive login shell"). In contrast, if you use
<tt>Session.execCommand()</tt> to start a command then (unless you ask for it) no pty will be aquired
and the command will be given to the shell as an argument (with the shell's "-c" option).
</p>
<p>
The way a shell is being invoked has an effect on the set of initialization files which will be read be the shell.
</p>
<p>
To demonstrate the difference, try the following (from the command line, e.g., with an OpenSSH client):
</p>
<p>
<ol>
<li>Login interactively and print the environment with the "env" command:<br> <br>
<tt><b>[user@host ~] ssh 127.0.0.1<br>
[user@host ~] env</b></tt><br> <br>
</li>
<li>Let the ssh server execute the "env" command (equivalent to using <tt>Session.executeCommand()</tt>):<br> <br>
<tt><b>[user@host ~] ssh 127.0.0.1 "env"</b></tt>
</li>
</ol>
</p>
<p>
If you compare the two outputs, then you will (unless you have adjusted your shell's settings)
observe different environments.
</p>
<p>
<b>If you are interested in the details, then please read the <tt>INVOCATION</tt> section in man page
for the bash shell. You may notice that the definitions of "interactive" and "non-interactive"
(and combinations with "login") are little bit tricky.</b>
</p>
[<a href="#oben">TOP</a>]
<hr><a name="blocking"></a><h2>My program sometimes hangs when I only read output from stdout!
Or: can you explain me the story about the shared stdout/stderr window in the SSH-2 protocol?
Or: what is this "StreamGobbler" thing all about?</h2>
<p>
In the SSH-2 low level protocol, each channel (e.g., session) has a receive window. When the remote
SSH daemon has filled up our receive window, it must wait until we have consumed the input and are ready to accept new data.
</p>
<p>
Unfortunately, the SSH-2 protocol defines a shared window for stderr and stdout. As a consequence,
if, for example, the remote process produces a lot of stderr data and you never consume it, then after
some time the local receive window will be full and the sender is blocked. If you then try to read()
from stdout, your call will be blocked: there is no stdout data (locally) available and the SSH daemon
cannot send you any, since the receive window is full (you would have to read some stderr data first
to "free" up space in the receive window).
</p>
<p>
Fortunately, Ganymed SSH-2 uses a 30KB window - the above described scenario should be very rare.
</p>
<p>
Many other SSH-2 client implementations just blindly consume any remotely produced data into a buffer
which gets automatically extended - however, this can lead to another problem: in the extreme case
the remote side can overflow you with data (e.g., leading to out of memory errors).
</p>
<p>
What can you do about this?
</p>
<p>
<ol>
<li><b>Bad: Do nothing</b> - just work with stderr and stdout Inputstreams and hope that the 30KB
window is enough for your application.</li>
<li><b>Better, recommended for most users:</b> use two worker threads that consume remote stdout
and stderr in parallel. Since you probably are not in the mood to program such a thing, you can use
the StreamGobbler class supplied with Ganymed SSH-2. The Streamgobbler is a special InputStream that
uses an internal worker thread to read and buffer internally all data produced by another InputStream.
It is very simple to use:<br> <tt><b><pre>InputStream stdout = new StreamGobbler(mysession.getStdout());
InputStream stderr = new StreamGobbler(mysession.getStderr());</pre></b></tt>
You then can access stdout and stderr in any order, in the background the StreamGobblers will
automatically consume all data from the remote side and store in an internal buffer.</li>
<li><b>Advanced:</b> you are paranoid and don't like programs that automatically extend buffers
without asking you. You then have to implement a state machine. The condition wait facility offered by
<tt>Session.waitForCondition()</tt> is exactly what you need: you can use it to wait until either stdout
or stderr data has arrived and can be consumed with the two InputStreams. You can either use the return value
of <tt>Session.waitForCondition()</tt> or check with <tt>InputStream.available()</tt>
(for stdout and stderr) which InputStream has data available (i.e., a <tt>read()</tt> call will not block).
Be careful when wrapping the InputStreams, also do not concurrently call read() on the InputStreams while calling
<tt>Session.waitForCondition()</tt> (unless you know what you are doing).<br>Please have a look a the
<tt>SingleThreadStdoutStderr.java</tt> example.</li>
<li><b>The lazy way:</b> you don't mind if stdout and stderr data is being mixed into the same
stream. Just allocate a "dumb" pty and the server will hopefully not send you any data on the stderr
stream anymore. <b>Note:</b> by allocating a pty, the shell used to execute the command will probably
behave differently in terms of initialization (see also <a href="#env">this question</a>).</li>
</ol>
</p>
[<a href="#oben">TOP</a>]
<hr><a name="buffered"></a><h2>Why are the session's Input- and OutputStreams not buffered?</h2>
<p>
If you need it, then this library offers quite a raw type of access to the SSH-2 protocol stack.
Of course, many people don't need that kind of low level access. If you need buffered streams,
then you should the do the same thing as you would probably do with the streams of a TCP socket:
wrap them with instances of BufferedInputStream and BufferedOutputStream. In case you use
StreamGobblers for the InputStreams, then you don't need any additional wrappers, since the
StreamGobblers implement buffering already.
</p>
<p>
This code snippet will probably work well for most people:
</p>
<p>
<tt>
<pre>
InputStream stdout = new StreamGobbler(mysession.getStdout());
InputStream stderr = new StreamGobbler(mysession.getStderr());
OutputStream stdin = new BufferedOutputStream(mysession.getStdin(), 8192);
</pre>
</tt>
</p>
[<a href="#oben">TOP</a>]
<hr><a name="sessioncommands"></a><h2>Why can't I execute several commands in one single session?</h2>
<p>
If you use <tt>Session.execCommand()</tt>, then you indeed can only execute only one command per session.
This is not a restriction of the library, but rather an enforcement by the underlying SSH-2 protocol
(a <tt>Session</tt> object models the underlying SSH-2 session).
</p>
<p>
There are several solutions:
</p>
<p>
<ul>
<li><b>Simple: Execute several commands in one batch</b>, e.g., something like <tt>Session.execCommand("echo
Hello && echo again")</tt>.</li>
<li><b>Simple: The intended way: simply open a new session for each command</b> - once you have opened a
connection, you can ask for as many sessions as you want, they are only a "virtual" construct.</li>
<li><b>Advanced: Don't use <tt>Session.execCommand()</tt>, but rather aquire a shell with
<tt>Session.startShell()</tt></b>. See also <a href="#env">this question</a>.</li>
</ul>
</p>
[<a href="#oben">TOP</a>]
<hr><a name="sessionlimit"></a><h2>I cannot open more than 10 concurrent sessions (or SCP clients).</h2>
<p>
You are probably using OpenSSH. By looking at their source code you will find out that there
is a hard-coded constant called MAX_SESSIONS in the session.c file which is set to "10" by default.
This is a per connection limit. Unfortunately, it is not a run-time tunable parameter.
However, this limit has no effect on the number of concurrent port forwardings. Please note: this information
is based on the OpenSSH 4.3 release.
</p>
<p>
Possible solutions:
<ul>
<li>(a) Recompile your SSH daemon</li>
<li>(b) Try to live with this limit and keep the number of concurrent sessions <= 10.</li>
<li>(c) Distribute your sessions over multiple concurrent SSH connections.</li>
</ul>
</p>
<p>
Just for completeness: starting from release 210, the thrown exception may look as follows:<br>
<tt>
<pre>
java.io.IOException: Could not open channel (The server refused to open the channel (SSH_OPEN_ADMINISTRATIVELY_PROHIBITED, 'open failed'))
</pre>
</tt>
</p>
[<a href="#oben">TOP</a>]
<hr><a name="passwordauth"></a><h2>Password authentication fails, I get "Authentication method password
not supported by the server at this stage".</h2>
<p>
Many default SSH server installations are configured to refuse the authentication type "password".
Often, they only accept "publickey" and "keyboard-interactive". You have different options:
</p>
<p>
<ul>
<li><b>Enable password authentication.</b> E.g., in case of OpenSSH on Fedora, edit
<code>/etc/sshd/sshd_config</code> and change the value of "PasswordAuthentication" to "yes",
then send a HUP signal to the daemon so that it re-reads its configuration.</li>
<li><b>Switch to public-key authentication.</b> Probably the best choice.</li>
<li><b>Try to use keyboard-interactive authentication.</b> If you have a GUI that interacts with a user,
then this is doable (check out the SwingShell.java example).</li>
</ul>
</p>
<p>
In general it is a good idea to call either <code>Connection.getRemainingAuthMethods()</code>
or <code>Connection.isAuthMethodAvailable()</code> before using a certain authentication method.
</p>
<p>
Please note that most servers let you in after one successful authentication step. However, in rare cases
you may encounter servers that need several steps. I.e., if one of the <code>Connection.authenticateWithXXX()</code>
methods returns <code>false</code> and <code>Connection.isAuthenticationPartialSuccess()</code> returns
<code>true</code>, then further authentication is needed. For each step, to find out which authentication methods
may proceed, you can use either the <code>Connection.getRemainingAuthMethods()</code>
or the <code>Connection.isAuthMethodAvailable()</code> method. Again, please have a look into the
SwingShell.java example.
</p>
[<a href="#oben">TOP</a>]
<hr><a name="puttygen"></a><h2>Why does public key authentication fail with my putty key?</h2>
<p>
When using putty private keys (e.g., .ppk files) with public key authentication, you get a
"Publickey authentication failed" exception. The reason is that the library currently is not able to
directly handle private keys in the proprietary format used by putty. However, you can use the
"puttygen" tool (from the putty website) to convert your key to the desired format: load your key,
then go to the conversions menu and select "Save OpenSSH key" (which saves the key in openssl PEM format,
e.g., call it "private.pem").
</p>
[<a href="#oben">TOP</a>]
<hr><a name="catmethod"></a><h2>I am sending data to a remote file using the "cat" method, but not all data is being written.</h2>
<p>
Please read carefully the answer to the following <a href="#pumptoremote">question</a>.
</p>
[<a href="#oben">TOP</a>]
<hr><a name="pumptoremote"></a><h2>I want to pump data into a remote file, but the amount of data to be sent
is not known at the time the transfer starts.</h2>
<p>
The SCP protocol communicates the amount of data to be sent at the start of the transfer,
so SCP remains out of consideration. Possible other solutions:
<ul>
<li>Use the SFTP client. Recommended.</li>
<li>Execute "cat > filename.txt" on the remote side and pump the data into stdin. This method is NOT recommended (and won't work on Windows...).</li>
</ul>
</p>
<p>
Be careful if you use the "cat" approach, as it may happen that not all your data will be
written. If you close the stdin stream and immediatelly close the session (or the whole connection) then
some SSH servers do not send the pending data to the process being executed ("cat" in this case).
You have to wait until "cat" has received the EOF and terminates before closing the session. However,
waiting for the termination may not always work, since SSH servers sometimes "forget" to send the exit code
of the remote process. The following code MAY work:
</p>
<p>
<tt>
<pre>
Session sess = conn.openSession();
sess.execCommand("cat > test.txt");
OutputStream stdin = sess.getStdin();
... out.write(...) ... out.write(...) ...
/* The following flush() is only needed if you wrap the */
/* stdin stream (e.g., with a BufferedOutputStream). */
out.flush();
/* Now let's send EOF */
out.close();
/* Let's wait until cat has finished */
sess.waitForCondition(ChannelCondition.EXIT_STATUS, 2000);
/* Better: put the above statement into a while loop! */
/* In ANY CASE: read the Javadocs for waitForCondition() */
/* Show exit status, if available (otherwise "null") */
System.out.println("ExitCode: " + sess.getExitStatus());
/* Now its hopefully safe to close the session */
sess.close();
</pre>
</tt>
</p>
<p>
(Just a thought for another solution: execute <code>cat > test.txt && echo "FINISHED"</code>
and wait until you get "FINISHED" on stdout... - try it on your own risk =)
</p>
[<a href="#oben">TOP</a>]
<hr><a name="swingshell"></a><h2>Do you have an example for the usage of feature XYZ?</h2>
<p>
Please have at look at the examples section in the distribution, especially at the SwingShell.java example.
</p>
[<a href="#oben">TOP</a>]
<hr><a name="maven"></a><h2>Where is the official Maven repository?</h2>
<p>
We regulary get requests for a Maven repository. Please note that <b>there is no such thing as an official Ganymed SSH-2 for Java Maven repository</b>.
At the moment, we do not have the resources to support specific build systems (be it Maven or anything else). We know that others
have setup (and not maintained) such repositories. However, we believe that you should
download security related software only from a trusted source - in other words, download the precompiled .jar file from our website and add it to your
project. This generic approach will work with every java development enviroment and build system.
Last warning: please think twice before you use a foreign "repository" to "auto-update" security related components of your project.
</p>
[<a href="#oben">TOP</a>]
</body>
</html>
|
