tag:blogger.com,1999:blog-6550210983769626522023-11-15T21:42:10.606+03:00Sergey Ushakov - IT blogSergey Ushakovhttp://www.blogger.com/profile/13035264684517536678noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-655021098376962652.post-82365387273399303752013-11-25T01:09:00.000+04:002017-10-21T12:40:55.211+03:00Yet another InstallCert for Java, now with STARTTLS support<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://www.oracle.com/technetwork/java/index.html"><img alt="Java" border="0" src="https://docs.oracle.com/javase/webdesign/other/im/Java_clr_hori.gif" title="Java" /></a></div>
<h4>
<a href="http://s-n-ushakov.blogspot.com/2013/11/yet-another-installcert-for-java-now.html#github">⇓ ... now forked at GitHub</a></h4>
<span style="font-size: smaller;">added on 2017-10-21</span><br />
<br />
<hr />
<br />
Many of the Java folks, who ever dealt with SSL-enabled protocols and self-made SSL certificates, know of the InstallCert tool. This simple command-line tool, published in 2006 by Andreas Sterbenz at the official Sun blog, allows obtaining SSL certificates as they are presented by the hosts we connect to, with further optional saving them to the system trust store.<br />
<br />
Sun's blog is not with us any more, but a copy of the original InstallCert publication and code is still available from some of the users' blogs, like <a href="http://nodsw.com/blog/leeland/2006/12/06-no-more-unable-find-valid-certification-path-requested-target">this one</a>, or archives like <a href="https://code.google.com/archive/p/misc-utils/wikis/JavaHttpsUrl.wiki">that one</a>. Curiously, one of the current blogs at Oracle <a href="https://blogs.oracle.com/gc/entry/unable_to_find_valid_certification">mentions</a> this tool, but without reference to the original author, and with a reference (currently somewhat outdated) to the mentioned user blog instead... <span style="font-size: 130%; line-height: 77%;">☺</span><br />
<br />
Well, the original Andreas' tool served faithfully to me for quite a while, but every good thing has its limitations... In particular, the original InstallCert could not deal with hosts that operate using <a href="https://en.wikipedia.org/wiki/STARTTLS">STARTTLS</a> technique.<br />
<br />
<h4>
The new code</h4>
<br />
Diving into STARTTLS required quite a refactoring of the original code, though the main parts of it are still in place <span style="font-size: 130%; line-height: 77%;">☺</span>. In particular, modular approach was taken to deal with STARTTLS implementations for different protocols, so the code does not fit in single Java file any more, but is rather packaged as an executable .jar.<br />
<br />
It is now possible to obtain certificates from hosts that not only speak plain SSL/TLS, but also expose their certificates via STARTTLS over IMAP, POP3, SMTP and LDAP.<br />
<br />
For new application-level protocols with STARTTLS extension to be supported, an abstract STARTTLS handler is defined as a <a href="http://www.usn.pp.ru/downloads/java/jsse-util-installcert/20140115/javadocs/usn/net/ssl/util/StarttlsHandler.html"><code>StarttlsHandler</code></a> interface. This interface needs to be implemented by every new protocol handler, and the latter is to be registered with the <a href="http://www.usn.pp.ru/downloads/java/jsse-util-installcert/20140115/javadocs/usn/net/ssl/util/Starttls.html"><code>Starttls</code></a> wrapper class. This registration needs to be hard-coded so far. But keeping in mind the small number of STARTTLS-compatible application-level protocols yet to be implemented, this should not be a problem <span style="font-size: 130%; line-height: 77%;">☺</span>.<br />
<br />
The certificates collected by the program are now stored at two locations:<br />
<ul style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>the standard <code>jssecacerts</code> keystore in the <code>lib/security</code> folder of your JRE;</li>
<li>in an <code>extracerts</code> keystore in your current directory; the latter may be handy in order to save collected certificates in pure form for further redistribution.</li>
</ul>
One of the new features is also the new template for collected certificates' aliases. They are now named like "host - certificate_subject" for better human readability <span style="font-size: 130%; line-height: 77%;">☺</span>.<br />
<br />
<h4>
Downloads</h4>
<br />
The following downloads are available:<br />
<ul style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>binary:
<ul style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li><a href="http://www.usn.pp.ru/downloads/java/jsse-util-installcert/20140115/export/installcert-usn-20140115.zip">http://www.usn.pp.ru/downloads/java/jsse-util-installcert/20140115/export/installcert-usn-20140115.zip</a></li>
<li><a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7Z1hxc3hMMGo0NWc">https://drive.google.com/uc?export=download&id=0B2nLiheurgV7Z1hxc3hMMGo0NWc</a></li>
</ul>
</li>
<li>source:
<ul style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li><a href="http://www.usn.pp.ru/downloads/java/jsse-util-installcert/20140115/export/installcert-usn-20140115-full.zip">http://www.usn.pp.ru/downloads/java/jsse-util-installcert/20140115/export/installcert-usn-20140115-full.zip</a></li>
<li><a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7WU9lOU9tdXpPR2c">https://drive.google.com/uc?export=download&id=0B2nLiheurgV7WU9lOU9tdXpPR2c</a></li>
</ul>
</li>
</ul>
Please feel free to use and modify. The <a href="http://www.usn.pp.ru/downloads/java/jsse-util-installcert/NOTICE.txt">original license</a> looks like <a href="http://opensource.org/licenses/BSD-3-Clause">3-clause BSD</a> one.<br />
<br />
<h4>
Usage – HOW-TO</h4>
<br />
<h5>
Prerequisites</h5>
<br />
<ol style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>Download the <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7Z1hxc3hMMGo0NWc"-->binary distribution archive from <a href="http://www.usn.pp.ru/downloads/java/jsse-util-installcert/20140115/export/installcert-usn-20140115.zip">here</a> or <a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7Z1hxc3hMMGo0NWc">here</a>.</li>
<li>Unzip it to a location of your choice.</li>
</ol>
<br />
<h5>
Obtaining a certificate from a plain SSL/TLS or an LDAP/STARTTLS server</h5>
<br />
Run the program like this:<br />
<pre style="background-color: #eeeeee; margin-left: 2em;">java -jar installcert-usn-20140115.jar <i>host_name</i>
</pre>
or<br />
<pre style="background-color: #eeeeee; margin-left: 2em;">java -jar installcert-usn-20140115.jar <i>host_name</i>:<i>port</i>
</pre>
or<br />
<pre style="background-color: #eeeeee; margin-left: 2em;">java -jar installcert-usn-20140115.jar <i>host_name</i>:<i>port</i> <i>truststore_password</i>
</pre>
<br />
The default port is 443 (for HTTPS). The default truststore password is "changeit" as per <a href="http://docs.oracle.com/javase/7/docs/technotes/tools/windows/keytool.html#cacerts">JSSE convention</a>.<br />
<br />
<h5>
Obtaining a certificate from an IMAP / POP3 / SMTP server with STARTTLS extension</h5>
<br />
In this case you will need the <a href="https://javaee.github.io/javamail/">JavaMail</a> library, and make sure you have it on your classpath. Please also keep in mind that it is necessary to indicate the main class explicitly in the command line if you have more than one jar.<br />
<br />
To make things easier, two shell scripts are provided: <code>run-with-javamail-starttls.sh</code> for Unix and <code>run-with-javamail-starttls.cmd</code> for Windows. You will have to edit at least one of them first, so to reflect the actual location of the JavaMail .jar file.<br />
<br />
<h5>
General notes and final housekeeping</h5>
<br />
If the program succeeds in obtaining a certificate (or
several of them), and the certificates are not known yet, it will ask
you whether you wish to save them. Upon successful run the program saves
the new certificate(s) to two files, as mentioned above:<br />
<ul style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>the standard <code>jssecacerts</code> keystore in the <code>lib/security</code> folder of your JRE;</li>
<li>an <code>extracerts</code> keystore in your current directory;
this one may be handy in order to save collected certificates in pure
form for further redistribution.</li>
</ul>
The first one will be needed by your software for normal JSSE operation. The second one is a good candidate for "clean" storage of your selected certificates.<br />
<br />
Please keep in mind that in order to have the standard <code>jssecacerts</code> keystore file in the <code>lib/security</code> folder of your JRE successfully created/modified, you will most likely need to have administrative (superuser, root) privileges.<br />
<br />
Enjoy! <span style="font-size: 130%; line-height: 77%;">☺</span><br />
<br />
<hr />
<br />
<h4>
<!-- looks like the 'Blogger' engine does not tolerate 'a' elements without a 'href' attribute and always adds something by its discretion, so we have to add something reasonable by ourselves before that... :) -->
<a href="javascript:;" name="tomcat8-implementation" style="color: inherit; cursor: inherit; text-decoration: inherit;">... rebuilt for Java 1.6 and fixed</a></h4>
<span style="font-size: smaller;">added on 2014-01-15</span><br />
<br />
The download links and examples were updated to reflect the new build made to be compatible with Java 1.6 as per <a href="http://s-n-ushakov.blogspot.ru/2013/11/yet-another-installcert-for-java-now.html?showComment=1389194071132#c8681698946163127104">Eric's comment</a>. The certificate handling logic was also improved for better discrimination of new certificates vs known ones.
<br />
<hr />
<br />
<h4>
<!-- looks like the 'Blogger' engine does not tolerate 'a' elements without a 'href' attribute and always adds something by its discretion, so we have to add something reasonable by ourselves before that... :) -->
<a href="javascript:;" id="github" style="color: inherit; cursor: inherit; text-decoration: inherit;">... now forked at GitHub</a></h4>
<span style="font-size: smaller;">added on 2017-10-21</span><br />
<br />
This code is now forked and available at Github: <a href="https://github.com/spyhunter99/installcert">https://github.com/spyhunter99/installcert</a> .
The fork was created <a href="https://stackoverflow.com/questions/7084482/how-to-save-the-ldap-ssl-certificate-from-openssl/20280562#comment78250553_20280562">with an intention to achieve embeddability</a>, and looks actively developed...Sergey Ushakovhttp://www.blogger.com/profile/13035264684517536678noreply@blogger.com22tag:blogger.com,1999:blog-655021098376962652.post-63655010875872537882013-06-17T09:14:00.000+04:002017-10-21T12:40:37.327+03:00Look Ma, no ResourceBundle :) Step 2: Dealing with message arguments<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://www.oracle.com/technetwork/java/index.html"><img alt="Java" border="0" src="https://docs.oracle.com/javase/webdesign/other/im/Java_clr_hori.gif" title="Java" /></a></div>
In the <a href="http://s-n-ushakov.blogspot.com/2013/02/look-ma-no-resourcebundle-or-yet.html">previous post</a> an approach to internationalization of Java applications was suggested that does not need a <a href="http://docs.oracle.com/javase/7/docs/api/java/util/ResourceBundle.html"><code>ResourceBundle</code></a> to do the job. Collections of localized messages may be arranged to be referenced as static class members rather than string aliases for properties in a <a href="http://docs.oracle.com/javase/7/docs/api/java/util/ResourceBundle.html"><code>ResourceBundle</code></a>. The main benefit is gained from getting references to messages compiler verifiable, with programmer's IDE autocompletion aids available for localized messages as a bonus <span style="font-size: 130%; line-height: 77%;">☺</span>.<br />
<br />
This approach was described and implemented previously at initial level, being still more of a concept than of a tool for real life applications. The main feature missing was an ability to supply arguments to localized messages, similar to <a href="http://docs.oracle.com/javase/7/docs/api/java/text/MessageFormat.html"><code>MessageFormat</code></a>.<br />
<br />
<h4>
Changes</h4>
<br />
Getting message arguments into the game resulted in certain refactoring, particularly in substantial separation of code from data and coining a concept of <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html"><code>I18nHandler</code></a>. Another change in concept was due to the fact that it is not possible to get a string value of an object via its <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#toString%28%29"><code>toString()</code></a> method if you are to supply formatting arguments. Hence, only methods like <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem2.html#s%28TArg1,%20TArg2%29"><code>s(TArg1 arg1, TArg2 arg2)</code></a> are available in these cases. Some less important change is that <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> no more extends <a href="http://docs.oracle.com/javase/7/docs/api/java/util/HashMap.html"><code>HashMap</code></a> but rather has more <a href="http://docs.oracle.com/javase/7/docs/api/java/util/Map.html"><code>Map</code></a> instances as members <span style="font-size: 130%; line-height: 77%;">☺</span>. And the former <code>p(String language, String message)</code> method for instantiating multi-language messages was refactored to <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/LocalizedMessage.html#lm%28java.lang.String,%20java.lang.String%29"><code>lm(String localeTag, String message)</code></a> in a separate class. One more notable change is that differentiation of messages by language turned into differentiation by <a href="http://docs.oracle.com/javase/7/docs/api/java/util/Locale.html#toLanguageTag%28%29">locale language tag</a>. And finally, the package was renamed to <code>usn.i18n.nobundle</code>.<br />
<br />
<h4>
New design</h4>
<br />
<h5>
Data</h5>
<br />
<a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/LocalizedMessage.html#lm%28java.lang.String,%20java.lang.String%29"><code>LocalizedMessage</code></a> is the basic container for localized messages. Internally it is a tuple of a <a href="http://docs.oracle.com/javase/7/docs/api/java/util/Locale.html#toLanguageTag%28%29">locale language tag</a> and a corresponding localized message. In case you do not care about locale-specific formatting of dates and numbers, feel free to reduce the full language tag to just the <a href="http://www.iana.org/assignments/language-subtag-registry">language subtag</a> like "en". The localized message may be a mere string or a formatting pattern as per <a href="http://docs.oracle.com/javase/7/docs/api/java/text/MessageFormat.html"><code>MessageFormat</code></a> convention. The standard way to construct a <code>LocalizedMessage</code> instance is via its static factory method <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/LocalizedMessage.html#lm%28java.lang.String,%20java.lang.String%29"><code>lm(String localeTag, String message)</code></a>. Normally there is no need to construct an isolated <code>LocalizedMessage</code> instance unless as a component of an <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a>.<br />
<br />
<a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> is the basic class to represent an internationalized message. Internally is has a collection of <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/LocalizedMessage.html#lm%28java.lang.String,%20java.lang.String%29"><code>LocalizedMessage</code></a> instances for a necessary set of locales. <code>I18nItem</code> is not intended for direct use by applications but rather serves as the base class for an hierarchy of more specific subclasses.<br />
<br />
Special care was taken to reduce the risk of wrong use of arguments for messages, e.g. providing arguments of wrong type or in wrong number. With this in mind the following <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> subclasses are offered for use:<br />
<ul>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem0.html"><code>I18nItem0</code></a> – the class to use for messages without arguments;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem1.html"><code>I18nItem1<TArg1></code></a> – a generic class to use for messages with exactly one argument;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem2.html"><code>I18nItem2<TArg1,TArg2></code></a> – a generic class to use for messages with exactly two arguments;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem3.html"><code>I18nItem2<TArg1,TArg2,TArg3></code></a> – a generic class to use for messages with exactly three arguments;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItemAny.html"><code>I18nItemAny</code></a> – a class to use for messages with arbitrary number of arguments, as the last resort for cases that did not fit into previous subclasses;</li>
</ul>
A snippet for declaring some set of internationalized messages might look like this:<br />
<pre style="background-color: #eeeeee;"><span style="color: #007700;">...</span>
<span style="color: #770077;">import</span> usn.i18n.nobundle.I18nItem;
<span style="color: #007700;">// get 'lm()' available as shorthand method without being prefixed by</span>
<span style="color: #007700;">// 'LocalizedMessage'</span>
<span style="color: #770077;">import static</span> usn.i18n.nobundle.LocalizedMessage.lm;
<span style="color: #007700;">...</span>
<span style="color: #770077;">static</span> I18nItem0 <span style="color: #000077;">MESSAGE_1</span> = <span style="color: #770077;">new</span> I18nItem0
(lm (<span style="color: blue;">"en"</span>, <span style="color: blue;">"That's cool!"</span>),
lm (<span style="color: blue;">"fr"</span>, <span style="color: blue;">"C'est le pied!"</span>),
lm (<span style="color: blue;">"ru"</span>, <span style="color: blue;">"Круто!"</span>));
<span style="color: #770077;">static</span> I18nItem1<Integer> <span style="color: #000077;">MESSAGE_2</span> = <span style="color: #770077;">new</span> I18nItem1<Integer>
(lm (<span style="color: blue;">"en"</span>, <span style="color: blue;">"I know {0,number,integer} guys seeking something"</span> +
<span style="color: blue;">" like this..."</span>),
lm (<span style="color: blue;">"fr"</span>, <span style="color: blue;">"Je sais que {0,number,integer} gars à la recherche"</span> +
<span style="color: blue;">" de quelque chose comme ceci..."</span>),
lm (<span style="color: blue;">"ru"</span>, <span style="color: blue;">"Я знаю ещё {0,number,integer} ребятишек, которые"</span> +
<span style="color: blue;">" хотели чего-то подобного..."</span>));
</pre>
<br />
<h5>
Handlers</h5>
<br />
The next important topic to take care of is an approach to obtaining user language / locale preferences during run time. In anticipation of vast variety of application architectures, this topic is resolved in the most general way with the help of the <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html"><code>I18nHandler</code></a> class. This class has absorbed almost all the code that is responsible for handling of methods like <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem2.html#s%28TArg1,%20TArg2%29"><code>s(TArg1 arg1, TArg2 arg2)</code></a>, and meanwhile it allows applications to achieve necessary behavior by its subclassing and overriding <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html#getDefaultLocaleTag%28%29"><code>getDefaultLocaleTag()</code></a> and most importantly <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html#getUserLocaleTags%28%29"><code>getUserLocaleTags()</code></a> methods. The only thing an application needs to do is:<br />
1) subclass <code>I18nHandler</code> or select from its existing subclasses as necessary;<br />
2) create an instance of the selected <code>I18nHandler</code> subclass; this instance will establish itself as a singleton and take care of all the rest.<br />
<br />
<a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandlerForSingleUser.html"><code>I18nHandlerForSingleUser</code></a> is an almost ready to use <code>I18nHandler</code> subclass that is intended for use by applications like desktop ones, with single user per application instance, when it is possible to define all user preferences upon application startup. You may only still wish to override the <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html#getDefaultLocaleTag%28%29"><code>getDefaultLocaleTag()</code></a> method.<br />
<br />
<h5>
Context based internationalization</h5>
<br />
There are also cases to be taken care of specially, when user preferences are conveniently available from some execution context rather than from the application directly. A standard example is a web application, where some primitive user preferences may be retrieved from a <a href="http://docs.oracle.com/javaee/6/api/javax/servlet/ServletRequest.html"><code>ServletRequest</code></a> instance. Or maybe even better from an associated <a href="http://docs.oracle.com/javaee/6/api/javax/servlet/http/HttpSession.html"><code>HttpSession</code></a> instance. These cases are taken care of by <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandlerInContext.html"><code>I18nHandlerInContext<TContext></code></a> and <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandlerForServletRequest.html"><code>I18nHandlerForServletRequest</code></a> subclasses, to be subclassed further as necessary.<br />
<br />
And <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandlerInContext.html"><code>I18nHandlerInContext<TContext></code></a> subclasses require a special family of <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> subclasses, that take a <code>TContext</code> instance as the first argument for all their methods:<br />
<ul>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItemInContext0.html"><code>I18nItemInContext0<TContext></code></a>;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItemInContext1.html"><code>I18nItemInContext1<TContext,TArg1></code></a>;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItemInContext2.html"><code>I18nItemInContext2<TContext,TArg1,TArg2></code></a>;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItemInContext3.html"><code>I18nItemInContext3<TContext,TArg1,TArg2,TArg3></code></a>;</li>
<li><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItemInContextAny.html"><code>I18nItemInContextAny<TContext></code></a>.</li>
</ul>
<br />
<h5>
Internationalization for logging</h5>
<br />
Finally, to bring the renovated package closer to real world application needs, two more classes are added that get this internationalization technology available for logging: <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I15dLogger.html"><code>I15dLogger</code></a> and <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I15dLoggerFactory.html"><code>I15dLoggerFactory</code></a>. These two bridge <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> with <a href="http://www.slf4j.org/">SLF4J</a> in hope that is going to be sufficient these days <span style="font-size: 130%; line-height: 77%;">☺</span>.<br />
<br />
<h4>
Behind the scenes</h4>
<br />
So what happens when an <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> instance's <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem0.html#s%28%29"><code>s()</code></a> or similar method is called?<br />
<br />
<ol style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>The <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html"><code>I18nHandler</code></a> singleton is located.</li>
<li>Its <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html#getUserLocaleTags%28%29"><code>getUserLocaleTags()</code></a> method is called. This method is expected to be overridden by a subclass to do something meaningful <span style="font-size: 130%; line-height: 77%;">☺</span>. An array of user preferred locale language tags is the result.</li>
<li>A negotiation between user preferences and locales available for given <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> instance takes place in the handler's <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html#findBestLocaleTag%28usn.i18n.nobundle.I18nItem,%20java.lang.String[]%29"><code>findBestLocaleTag(I18nItem,String[])</code></a> method. For every user's preferred locale two attempts are made: for exact match and for approximate match via language subtag. If no luck, then the <a href="http://docs.oracle.com/javase/7/docs/api/java/util/Locale.html#getDefault%28%29">system default locale</a> is attempted. If again no luck, then plain English is attempted. And if not successful again, the first locale available for given <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> instance is selected.</li>
<li>The localized message (message pattern) is retrieved from given <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem.html"><code>I18nItem</code></a> instance for the locale language tag selected on the previous step.</li>
<li>Message formatting arguments are applied if required.</li>
</ol>
<br />
<h4>
Downloads</h4>
<br />
Both <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7VmpsSjE0MWItRzA"--><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/export/usn-i18n-nobundle-20130617.jar">jar</a> and <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7b25aNGk5SmNfYmc"--><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/export/usn-i18n-nobundle-full-20130617.zip">source</a> downloads are available. <span style="background-color: #ffffaa;" title="edit of 2015-07-23">And the project is now <a href="https://github.com/s-n-ushakov/usn-i18n-nobundle">hosted at Github</a>.</span> Please feel free to use and modify. The license is <a href="http://opensource.org/licenses/BSD-2-Clause">BSD</a>.<br />
<br />
<br />
<h4>
Usage – HOW-TO</h4>
<br />
<ol style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>Download the <a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7VmpsSjE0MWItRzA">.jar file</a> <span style="font-size: 130%; line-height: 77%;">☺</span>.</li>
<li>Add it to your application's class path.</li>
<li>Make a decision on the <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html"><code>I18nHandler</code></a> subclass you need: whether it should depend on some context for obtaining user's locale preferences or should it get these preferences from the application directly; hence select an existing <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html"><code>I18nHandler</code></a> subclass or make a subclass of your own.</li>
<li>Create an instance of the chosen <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nHandler.html"><code>I18nHandler</code></a> subclass in your application; no need to care about its further fate; feel free to mark it with <code>@SuppressWarnings ("unused")</code>.</li>
<li>Create as many of <code>I18nItem</code> sublasses' instances as you need for all the messages that require internationalization.<br />
</li>
<li>Use your messages via corresponding <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/I18nItem1.html#s%28TArg1%29"><code>s(TArg1 arg1)</code></a> and similar methods.</li>
<li>Refer to the <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/usn/i18n/nobundle/package-summary.html#package_description">modest example on the package summary javadocs page</a> and <a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130617/javadocs/index.html">to API docs</a> in general when necessary.</li>
<li>Enjoy <span style="font-size: 130%; line-height: 77%;">☺</span>.</li>
</ol>
<br />
<h4>
Package name</h4>
<br />
I personally feel not very comfortable having a negative word like "nobundle" in the package name. But got no better idea so far. So positive ideas on naming are welcome! <span style="font-size: 130%; line-height: 77%;">☺</span><br />
<br />
<h4>
Comments welcome</h4>
<br />
Though the library is already tested in a real life application, it is probably undercooked <span style="font-size: 130%; line-height: 77%;">☺</span> and may contain bugs. It may also ask for improvement here and there. So any comments are welcome too!.. <span style="font-size: 130%; line-height: 77%;">☺</span>Sergey Ushakovhttp://www.blogger.com/profile/13035264684517536678noreply@blogger.com0tag:blogger.com,1999:blog-655021098376962652.post-70104020380765594952013-02-23T11:06:00.002+04:002017-10-21T12:40:08.500+03:00Look Ma, no ResourceBundle :) Or yet another approach to i18n in Java<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://www.oracle.com/technetwork/java/index.html"><img alt="Java" border="0" src="https://docs.oracle.com/javase/webdesign/other/im/Java_clr_hori.gif" title="Java" /></a></div>
Everyone knows the traditional way to make a Java application international: one should give every text message a symbolic name, place all the strings in various languages in a set of special resource files, and then get message strings used via their symbolic names in the <a href="http://docs.oracle.com/javase/7/docs/api/java/util/ResourceBundle.html">prescribed manner</a>. This is not all of course, and one furthermore has to face different locales, plurals, genders, etc. ... And <a href="http://docs.oracle.com/javase/7/docs/api/java/util/ResourceBundle.html"><code>ResourceBundle</code></a> has always been an important part of these internationalization facilities.<br />
<br />
This approach is well developed, and lots of tools are available to help using it, and still there is an issue about it. Java code is linked to string resources via symbolic names that are mere strings, and it is pretty easy to get them out of sync in the course of changes. And as these links are not checked by compiler, a programmer is almost certain to feel happy until a surprise at run time...<br />
<br />
A colleague of mine has once mentioned that it would be more robust and convenient to have text resources linked to Java code via class member names. Then these links would be checked by the compiler, and a programmer could also benefit from autocompletion facilities provided by modern IDE's.<br />
<br />
It took some time to get the idea crystallized, and here is the result. It is all about a class named <code>I18nItem</code> that extends <a href="http://docs.oracle.com/javase/7/docs/api/java/util/HashMap.html"><code>HashMap</code></a> (that might be any other <a href="http://docs.oracle.com/javase/7/docs/api/java/util/Map.html"><code>Map</code></a>) and has an overridden <a href="http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#toString%28%29"><code>toString()</code></a> method. Every instance of such a class would wrap one text message in different languages, indexed by language. One more task was to implement a convenient way to construct such collections as members of some class, so to allow easy referring to them from other parts of code.<br />
<br />
<h4>
The new approach in action</h4>
<br />
Let's have a look at what a standard use case might look like.<br />
<br />
First an application would typically have <code>I18nItem</code> subclassed:<br />
<pre style="background-color: #eeeeee;"><span style="color: #007700;">...</span>
<span style="color: #770077;">import</span> usn.util.i18n.I18nItem;
<span style="color: #770077;">public class</span> AppSpecificI18nItem <span style="color: #770077;">extends</span> I18nItem
{
<span style="color: #770077;">private static final long</span> <span style="color: #000077;">serialVersionUID</span> = 1L;
<span style="color: #007700;">// define a simple subclass constructor to be used</span>
AppSpecificI18nItem (I18nItem.LanguagePair... data)
{
<span style="color: #770077;">super</span> (data);
}
<span style="color: #007700;">// let's assume the default application language is going to be</span>
<span style="color: #007700;">// French, so override the appropriate method</span>
<span style="color: #777777;">@Override</span>
<span style="color: #770077;">protected</span> String getDefaultLanguage ()
{
<span style="color: #770077;">return</span> <span style="color: blue;">"fr"</span>;
}
<span style="color: #007700;">// then the application needs to define the way to obtain current</span>
<span style="color: #007700;">// user's language preferences as string array, most preferred</span>
<span style="color: #007700;">// ones coming first</span>
<span style="color: #777777;">@Override</span>
<span style="color: #770077;">protected</span> String [] getUserLanguages ()
{
<span style="color: #007700;">// normally we are expected to do something more interesting</span>
<span style="color: #007700;">// here...</span>
<span style="color: #770077;">return</span> <span style="color: #770077;">super</span>.getUserLanguages ();
}
} <span style="color: #007700;">// class AppSpecificI18nItem</span>
</pre>
<br />
Next we need to construct our international string resources without using <code>ResourceBundle</code> <span style="font-size: 130%; line-height: 77%;">☺</span> – in any Java class you may find appropriate:
<br />
<pre style="background-color: #eeeeee;"><span style="color: #007700;">...</span>
<span style="color: #770077;">import</span> usn.util.i18n.I18nItem;
<span style="color: #007700;">// get 'p()' available as shorthand method without being prefixed by</span>
<span style="color: #007700;">// 'I18nItem'</span>
<span style="color: #770077;">import static</span> usn.util.i18n.I18nItem.p;
<span style="color: #770077;">public class</span> MyMessages
{
<span style="color: #770077;">public static</span> I18nItem <span style="color: #000077;">MESSAGE_1</span> = <span style="color: #770077;">new</span> AppSpecificI18nItem
(p (<span style="color: blue;">"en"</span>, <span style="color: blue;">"That's cool!"</span>),
p (<span style="color: blue;">"fr"</span>, <span style="color: blue;">"C'est le pied!"</span>),
p (<span style="color: blue;">"ru"</span>, <span style="color: blue;">"Круто!"</span>));
<span style="color: #770077;">public static</span> I18nItem <span style="color: #000077;">MESSAGE_2</span> = <span style="color: #770077;">new</span> AppSpecificI18nItem
(p (<span style="color: blue;">"en"</span>, <span style="color: blue;">"Just what we need!"</span>),
p (<span style="color: blue;">"fr"</span>, <span style="color: blue;">"Exactement ce qu'il nous faut!"</span>),
p (<span style="color: blue;">"ru"</span>, <span style="color: blue;">"То, что нужно!"</span>));
<span style="color: #007700;">// ...</span>
} <span style="color: #007700;">// class MyMessages</span>
</pre>
<br />
And finally we are going to use the messages we have defined:
<br />
<pre style="background-color: #eeeeee;"><span style="color: #007700;">...</span>
<span style="color: #007700;">// in some cases implicit conversion to String is available, and the</span>
<span style="color: #007700;">// overridden 'getUserLanguages()' method combined with</span>
<span style="color: #007700;">// 'I18nItem#toString()' takes care of everything...</span>
System.<span style="color: #000077;">out</span>.println (MyMessages.<span style="color: #000077;">MESSAGE_1</span>);
<span style="color: #007700;">// ... and in other cases the 's()' shorthand method is at our</span>
<span style="color: #007700;">// disposal...</span>
logger.info (MyMessages.<span style="color: #000077;">MESSAGE_2</span>.s ());
</pre>
<br />
And in every case like these your IDE is likely to assist you with right selection of the message to use...<br />
<br />
<h4>
Downloads</h4>
<br />
Both <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7R0RZZEV3MFJIa00"--><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130218/usn-i18n-util-full-20130218.zip">jar-with-source</a> and <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7OWlQMUdtMlk3U3M"--><a href="http://www.usn.pp.ru/downloads/java/usn-i18n-nobundle/20130218/usn-i18n-util-demo-20130218.zip">demo</a> downloads are available. Please feel free to use and modify. The license is <a href="http://opensource.org/licenses/BSD-2-Clause">BSD</a>.<br />
<br />
<h4>
Acknowledgements</h4>
<br />
Special thanks to my colleague <a href="http://www.facebook.com/s.a.petrov.spb">Sergey Petrov</a> for the idea that access to a resource via some class member might be preferable against string alias <span style="font-size: 130%; line-height: 77%;">☺</span>.<br />
<br />
<hr />
<br />
<h4>
<a href="http://s-n-ushakov.blogspot.ru/2013/06/look-ma-no-resourcebundle-step-2.html">Continued: Step 2: Dealing with message arguments</a></h4>
<span style="font-size: smaller;">added on 2013-06-21</span>Sergey Ushakovhttp://www.blogger.com/profile/13035264684517536678noreply@blogger.com0tag:blogger.com,1999:blog-655021098376962652.post-35408789359947364812011-09-30T08:59:00.006+04:002017-10-21T12:39:49.245+03:00XSLT, entities, Java, Xalan...<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://www.oracle.com/technetwork/java/index.html"><img alt="Java" border="0" src="https://docs.oracle.com/javase/webdesign/other/im/Java_clr_hori.gif" title="Java" /></a></div>
<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://xml.apache.org/xalan-j/"><img alt="The Apache XML Project / Xalan-J" border="0" src="https://xml.apache.org/images/group-logo.gif" title="The Apache XML Project / Xalan-J" width="172" /></a></div>
As already mentioned, I have been using <a href="http://www.w3.org/TR/xslt">XSLT</a> and <a href="http://xml.apache.org/xalan-j/">Xalan-J</a> for quite a long while. It was recently that I was dealing with an HTML-related transform that used a lot of special characters.<br />
<br />
Previously I blindly used to assume that <a href="http://www.w3.org/TR/html5/named-character-references.html">named character references</a> are a part of HTML, not defined for generic XML, so not available in XSLT files too. No big problem if <a href="http://www.w3.org/TR/html5/syntax.html#character-references">numeric character references</a> are available. But that time I had a good incentive to give it a better thought <span style="font-size: 130%; line-height: 77%;">☺</span>. After all, the very name of XML states extensibility. So what prevents us from getting named character references being defined for XSLT files too?<br />
<br />
In fact, nothing prevents. An absolutely legal way to get them defined is to extend the <a href="http://www.w3.org/TR/xml/#sec-prolog-dtd">document type declaration</a> for XSLT, like this:<br />
<pre style="background-color: #eeeeee;"><?xml version="1.0" standalone="yes" ?>
<!DOCTYPE transform [
<!ENTITY % w3centities-f PUBLIC "-//W3C//ENTITIES Combined Set//EN//XML"
"http://www.w3.org/2003/entities/2007/w3centities-f.ent">
%w3centities-f;
]>
<xsl:transform version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
...
</pre>
<br />
With this small modification the miracle happened, and named character references stopped complaining <span style="font-size: 130%; line-height: 77%;">☺</span>. Everything was fine, but, as one of the <a href="http://www.murphys-laws.com/murphy/murphy-laws.html">Murphy's laws</a> states, "If everything seems to be going well, you have obviously overlooked something" <span style="font-size: 130%; line-height: 77%;">☺</span>.<br />
<br />
This time the pitfall was in the reference to the entities definition, <br />
<code>"http://www.w3.org/2003/entities/2007/w3centities-f.ent"</code>. If we tell the XSLT engine to use some external resource, it naturally has to go for it <span style="font-size: 130%; line-height: 77%;">☺</span>. In case we do not take necessary care, the only place to go is the Internet. Everything is going to work, but maybe slower than we might expect, and crashing if Internet connection is not available.<br />
<br />
The standard approach to deal with this is using a <a href="http://en.wikipedia.org/wiki/XML_Catalog">catalog-based entity/URL resolver</a>. This time unexpectedly it did not help. Nothing was wrong with the resolver and the catalog, and still the XSLT engine persistently went fetching the entities from the Internet.<br />
<br />
The cause of the issue was found in the Xalan-J sources. Perhaps nobody before considered seriously using external entities in an XSLT file, thus no traces of using an entity resolver for an XSLT file in the code. It may be worth mentioning that those of us who might be not familiar with Xalan-J as such and just plainly use <a href="http://download.oracle.com/javase/tutorial/jaxp/index.html">JAXP</a>, are likely to tread on the same issue, as Xalan-J is a part of the <a href="http://jaxp.java.net/1.4/JAXP-FAQ.html">reference JAXP implementation</a> that is endorsed into standard JDK/JRE implementations like <a href="http://www.oracle.com/technetwork/java/javase/downloads/index.html">Oracle J2SE</a> and <a href="http://openjdk.java.net/">OpenJDK</a>.<br />
<br />
<h4>
The Fix</h4>
<br />
The last Xalan-J release 2.7.1 was taken as the basis for amendments. For those who need just things working, a patched Xalan-J binary build is available as <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZTNmZjgyNmUtYWExMi00MzdmLWFmNzEtYjVmN2Q1Y2MyOWU2"--><a href="http://www.usn.pp.ru/downloads/java/apache-xalan-j/xalan-2.7.1-usn-20110928.jar">xalan-2.7.1-usn-20110928.jar</a>. Those interested in what is under the hood are welcome to have a look at the <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7NGViOGEwZTMtNTEzMS00NWJkLWJjZDQtY2U5OTliNmNlMTIy"--><a href="http://www.usn.pp.ru/downloads/java/apache-xalan-j/xalan-2.7.1-usn-20110928-modifications.zip">modifications</a>.<br />
<br />
The fix was submitted to the Xalan-J project as an attachment to issue <a href="https://issues.apache.org/jira/browse/XALANJ-2544">XALANJ-2544</a>.<br />
<br />
<h4>
Usage – HOW-TO</h4>
<br />
In case you feel like using some named character references in your XSLT file:<br />
<ol style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>Add appropriate entities to your XSLT transform/stylesheet, either one by one, like this:<br />
<pre style="background-color: #eeeeee;"><?xml version="1.0" standalone="yes" ?>
<!DOCTYPE transform [
<!ENTITY nbsp "&#x000A0;">
<!ENTITY ndash "&#x02013;">
]>
<xsl:transform version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
...
</pre>
or all at once:<br />
<pre style="background-color: #eeeeee;">...
<!DOCTYPE transform [
<!ENTITY % w3centities-f PUBLIC "-//W3C//ENTITIES Combined Set//EN//XML"
"http://www.w3.org/2003/entities/2007/w3centities-f.ent">
%w3centities-f;
]>
...
</pre>
</li>
<li>If you could limit yourself to a definite set of entities, or your XSLT engine is not Xalan-based, have fun <span style="font-size: 130%; line-height: 77%;">☺</span></li>
<li>If not, go ahead and download the patched Xalan-J binary as <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZTNmZjgyNmUtYWExMi00MzdmLWFmNzEtYjVmN2Q1Y2MyOWU2"--><a href="http://www.usn.pp.ru/downloads/java/apache-xalan-j/xalan-2.7.1-usn-20110928.jar">xalan-2.7.1-usn-20110928.jar</a>.</li>
<li>Arrange your JVM to use the patched .jar, keeping in mind that Xalan-J is an endorsed technology. One of possible approaches is to use <code>-Djava.endorsed.dirs=<i>path_to_the_folder_containing_the_new_jar</i></code> JVM launcher option. See more on the Oracle <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/standards/">Java Endorsed Standards Override Mechanism</a> page.</li>
<li>Make sure you use some <a href="http://xml.apache.org/commons/components/resolver/resolver-article.html">XML Entity Resolver</a>. You may wish to have a look at <a href="http://xml.apache.org/commons/components/resolver/">Apache XML Commons Resolver</a> as a good candidate.</li>
<li>Get yourself a catalog for your resolver, if not yet. If starting from scratch, that may be a plain text file in <a href="http://www.oasis-open.org/specs/a401.htm">OASIS TR9401</a> format:<br />
<pre style="background-color: #eeeeee;">PUBLIC "-//W3C//ENTITIES Combined Set//EN//XML"
"file:///<i>path_to_your_local_copy</i>/w3centities-f.ent"
SYSTEM "http://www.w3.org/2003/entities/2007/w3centities-f.ent"
"file:///<i>path_to_your_local_copy</i>/w3centities-f.ent"
</pre>
</li>
<li>Get your catalog known to the resolver, say, via the <code>xml.catalog.files</code> system property supplied to the JVM launcher: <code>-Dxml.catalog.files=<i>your_catalog_file_name_with_path</i></code>.</li>
<li>Enjoy <span style="font-size: 130%; line-height: 77%;">☺</span>. </li>
</ol>
Sergey Ushakovhttp://www.blogger.com/profile/13035264684517536678noreply@blogger.com1tag:blogger.com,1999:blog-655021098376962652.post-82962891796201917212011-09-04T11:12:00.006+04:002017-10-21T12:43:11.600+03:00Java, Xalan, Unicode command line arguments...<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://www.oracle.com/technetwork/java/index.html"><img alt="Java" border="0" src="https://docs.oracle.com/javase/webdesign/other/im/Java_clr_hori.gif" title="Java" /></a></div>
I have always believed on bare word that Java is Unicode-friendly by design. And I have been using <a href="http://xml.apache.org/xalan-j/">Xalan-J</a> happily for quite a long while already.<br />
<br />
It was just recently that I ran into Unicode-related problem with Java, and that was occasionally related to Xalan-J. Strictly speaking, Xalan-J was not to blame. But that was <a href="http://xml.apache.org/xalan-j/commandline.html">Xalan-J Command-Line Utility</a> who refused to do its job when handling file names with characters missing from my local system code page.<br />
<br />
It did not take a long search to discover that the problem is known and platform-specific, at least specific for Windows. The bug is known at least since the early days when the yet-to-come Java 6 was called Mustang (<a href="http://www.java.net/node/644569">"Support for Unicode in Mustang ?"</a> , 2005). It was discussed later on JavaRanch - <a href="http://www.coderanch.com/t/279253/Streams/java/Unicode-cmd-parameters-main-args#1284172">Unicode: cmd parameters (main args); exec parameters; filenames</a> and on OpenJDK 'core-libs-dev' mailing list - <a href="http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-March/thread.html#1231">RFE 4519026: (process) Process should support Unicode on Win NT, request for review</a> and <a href="http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-February/thread.html#1145">Unicode support in Java JRE on Windows</a>.<br />
<br />
The origin of the bug looks like dating back to the days when it was necessary to have Java running on both Unicode and non-Unicode (e.g. Windows 9x) platforms, so using non-Unicode system calls in Java launcher code was somehow justified. A lot of water has flown under the bridge since then, and Windows 9x is no more supported by new JVM versions... And still my quick experiment revealed that the new Java 7 comes with the same bug in its place...<br />
<br />
The right approach would be to get the bug fixed in the Java launcher. But unfortunately it is already long since I wrote something in C/C++... So the quick-and-dirty solution happened to be a Java wrapper.<br />
<br />
<h4>
Wrapper Implementation</h4>
<br />
The first idea was to get a wrapper just for Xalan-J. But then I remembered of the other command-line utilities written in Java and came to an idea of a general-purpose command line wrapper.<br />
<br />
It was necessary to provide possibility of handling Unicode parameters at command shell level (e.g. file names, etc). And then supply all the command line arguments to Java, bypassing the command line <span style="font-size: 130%; line-height: 77%;">☺</span>. The solution was found as passing the command line arguments via a temporary file, one argument per line, to be written by command shell in some flavor of Unicode, to be further read in by the Java wrapper. And there was no rich variety of convenient Unicode flavors under Windows, as the <code>cmd</code> tool only allows easy output in UTF-16LE using the <code>/u</code> switch.<br />
<br />
The resulting wrapper is available for download and free use under <a href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0</a> license, as both <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZGI0NGY3MzQtZTJhZi00NTY2LWEzMTAtM2E5YWJlMjMyMTU5"--><a href="http://www.usn.pp.ru/downloads/java/usn-unicode/usn-unicode-20110903-src.zip">source</a> and a compiled <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZTRiNGM5ZjgtOGI4Zi00NjUwLTlkNTAtOTAwNWI0OWI2ZWNj"--><a href="http://www.usn.pp.ru/downloads/java/usn-unicode/usn-unicode-20110903.jar">.jar</a>.<br />
<br />
Other command-line tools coming to my mind besides Xalan-J, that might benefit from using with this wrapper, include: <a href="http://xmlgraphics.apache.org/batik/tools/rasterizer.html">Batik</a>, <a href="http://xmlgraphics.apache.org/fop/1.0/running.html">FOP</a>, <a href="http://www.re.be/css2xslfo/index.xhtml">CSSToXSLFO</a>, ...<br />
<br />
<h4>
Usage – HOW-TO</h4>
<br />
<ol style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>Download the <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZTRiNGM5ZjgtOGI4Zi00NjUwLTlkNTAtOTAwNWI0OWI2ZWNj"--><a href="http://www.usn.pp.ru/downloads/java/usn-unicode/usn-unicode-20110903.jar">.jar file</a> and save it to a location of your choice.</li>
<li>Prepare / modify the script you use to launch your command-line tool. Say, for Xalan-J your script might contain something like:<br />
<pre style="background-color: #eeeeee;">@cmd /u /c echo -IN >%TMP%\xalan-args.tmp
@cmd /u /c echo %~f1 >>%TMP%\xalan-args.tmp
@cmd /u /c echo -XSL >>%TMP%\xalan-args.tmp
@cmd /u /c echo %~f2 >>%TMP%\xalan-args.tmp
@cmd /u /c echo -OUT >>%TMP%\xalan-args.tmp
@cmd /u /c echo %~f3 >>%TMP%\xalan-args.tmp
@if not %4. == . cmd /u /c echo %~4 >>%TMP%\xalan-args.tmp
@if not %5. == . cmd /u /c echo %~5 >>%TMP%\xalan-args.tmp
...
java [...] usn.unicode.UnicodeLauncher org.apache.xalan.xslt.Process %TMP%\xalan-args.tmp
</pre>
Ensure that <code>usn-unicode-20110903.jar</code> is on your class path. Note that all right angle redirection characters are doubled, except the first line... Refer to the <a href="http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/percent.mspx?mfr=true">"Using batch parameters" Microsoft page</a> for syntax like <code>%~f1</code>. Note also <code>%~4</code> instead of <code>%4</code> to remove surrounding quotes if any.<br />
</li>
<li>Have fun <span style="font-size: 130%; line-height: 77%;">☺</span></li>
</ol>
Sergey Ushakovhttp://www.blogger.com/profile/13035264684517536678noreply@blogger.com0tag:blogger.com,1999:blog-655021098376962652.post-47467002284960488242011-08-19T09:36:00.013+04:002017-10-21T12:49:51.840+03:00Tomcat Redirect Valve<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://www.oracle.com/technetwork/java/index.html"><img alt="Java" border="0" width="146" src="https://docs.oracle.com/javase/webdesign/other/im/Java_clr_hori.gif" title="Java" /></a></div>
<div class="separator" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;">
<a href="http://tomcat.apache.org/"><img alt="Apache Tomcat" border="0" src="https://tomcat.apache.org/tomcat-7.0-doc/images/tomcat.gif" title="Apache Tomcat" /></a></div>
<h4>
<a href="http://s-n-ushakov.blogspot.ru/2011/08/tomcat-redirect-valve.html#jboss-implementation">⇓ Probably the most important part – "All we need is already here"</a></h4>
<span style="font-size: smaller;">added on 2011-11-27</span><br />
<br />
<h4>
<a href="http://s-n-ushakov.blogspot.ru/2011/08/tomcat-redirect-valve.html#tomcat8-implementation">⇓ Another important news – ... now implemented as Tomcat 8 component</a></h4>
<span style="font-size: smaller;">added on 2013-03-17</span><br />
<br />
<h4>
<a href="http://s-n-ushakov.blogspot.ru/2011/08/tomcat-redirect-valve.html#tomcat8-release">⇓ And the final news – ... now available as standard Tomcat feature</a></h4>
<span style="font-size: smaller;">added on 2014-11-16</span><br />
<br />
<hr />
<br />
<h4>
Background</h4>
<br />
Every mature web resource eventually realizes that it needs <a href="http://en.wikipedia.org/wiki/URL_redirection">URL redirection</a>.<br />
<br />
Leaving aside manual and client-side redirection techniques, and concentrating on <a href="http://en.wikipedia.org/wiki/Server-side_redirect">server-side redirection</a>, one can see that in the <a href="http://httpd.apache.org/">Apache HTTP Server</a> world <a href="http://httpd.apache.org/docs/current/mod/mod_rewrite.html">mod_rewrite</a> is the canonical and powerful tool to do the job.<br />
<br />
In the Java world there is also a recognized leader for this task. That is <a href="http://www.tuckey.org/urlrewrite/">UrlRewriteFilter</a>, also <a href="http://code.google.com/p/urlrewritefilter/">hosted at Google Code</a>. BTW, the front page of that project contains a good list of reasons for one to be interested in URL rewriting (redirection).<br />
<br />
There is nothing wrong about UrlRewriteFilter. It is feature rich, and mature, and well supported, and actively developed. Nothing wrong except it is a <a href="http://download.oracle.com/javaee/6/api/javax/servlet/Filter.html">Filter</a> <span style="font-size: 130%; line-height: 77%;">☺</span>.<br />
<br />
On one hand, being a Filter is not bad. In any case Filter is a standard concept defined by the <a href="http://www.oracle.com/technetwork/java/javaee/servlet/index.html">Servlet </a>specification, so portability across different web application containers may be considered as guaranteed.<br />
<br />
On the other hand, having URL rewriting as part of a web application may be not always sufficient. It is not uncommon for a servlet container to host more than one web application, and thus separation of responsibilities may become a concern. Web application belongs to the developer's responsibility domain. Meanwhile the container belongs to the administrator's responsibility domain. And an administrator may have his own ideas on the requirements for redirection. So an administrator may need a separate facility for setting up server-side redirects securely, without risk of being blamed for ruining the developer's masterpiece... <span style="font-size: 130%; line-height: 77%;">☺</span> And speaking seriously, an administrator may occasionally wish to establish server-wide redirection policies that would be common for all the applications being hosted.<br />
<br />
Unfortunately there is no common standard for Java web application containers that we could base upon for this task. Every container seems to have its own specifics. Narrowing our needs to <a href="http://tomcat.apache.org/">Tomcat</a> as one of the widely used containers we may come to an idea that a <a href="http://tomcat.apache.org/tomcat-7.0-doc/config/valve.html">Valve</a> may be the right tool. BTW, there is an noteworthy consideration of <a href="http://blog.frankel.ch/tomcats-valve-an-alternative-to-filter">Valves vs Filters</a> in Nicolas Fränkel's blog.<br />
<br />
The idea of a Redirect Valve for Tomcat has been in the air for quite a while already. Curious to note, it was even on the <a href="http://svn.apache.org/repos/asf/tomcat/archive/tc4.0.x/trunk/container/catalina/docs/dev/todo.html">Tomcat 4 TODO list</a>. And deep googling for a "RedirectValve" reveals several in-house implementations of such a Valve being mentioned across various exception logs here and there. Meanwhile it looks like nobody cared to make such a tool publicly available so far...<br />
<br />
<h4>
Implementation</h4>
<br />
The first draft implementation of a Redirect Valve was <a href="https://issues.apache.org/bugzilla/show_bug.cgi?id=14766">suggested in 2002</a> by Jens Andersen. Unfortunately the implementation was incomplete and thus was not adopted for inclusion into the Tomcat project.<br />
<br />
Bringing the existing draft to working state was not a hard job. So the final result:<br />
<ul>
<li>allows configuration of a Valve instance in terms of <a href="http://download.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html">Java regular expressions</a> and substitutions using <a href="http://download.oracle.com/javase/7/docs/api/java/util/regex/Matcher.html#group%28int%29">capturing groups</a>;</li>
<li>for every incoming HTTP request:<br />
<ul>
<li>makes an effort to reconstruct the original request URL from sparse pieces available at the valve level;</li>
<li>saves the URL as a <a href="http://download.oracle.com/javaee/6/api/javax/servlet/ServletRequest.html#setAttribute%28java.lang.String,%20java.lang.Object%29">request attribute</a> to be used by subsequent RedirectValve instances that may follow in the pipeline;</li>
<li>matches the URL against a pre-configured regular expression;</li>
<li>in case the match succeeds, sets the response status code as <a href="http://download.oracle.com/javaee/6/api/javax/servlet/http/HttpServletResponse.html#SC_MOVED_PERMANENTLY">SC_MOVED_PERMANENTLY</a>, sets the response "Location" header as prescribed by a pre-configured expression using capturing groups and gets the request out of the processing pipeline;</li>
</ul>
</li>
</ul>
Well, nothing fancy at all, but looks like doing the job for standard cases...<br />
<br />
This RedirectValve implementation was <a href="https://issues.apache.org/bugzilla/show_bug.cgi?id=51468">submitted to the Tomcat project</a> again. The result was the same as for the submission of 2002, the reasons now being that in the times of <a href="http://www.tuckey.org/urlrewrite/">UrlRewriteFilter</a> there is no need for other tools, and those willing to have a redirection Valve are free to wrap UrlRewriteFilter into a Valve by themselves... <span style="font-size: 130%; line-height: 77%;">☺</span><br />
<br />
Admitting this may be true, I still dare to offer this Valve to the public. Please feel free to use and modify. The license is <a href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0</a>. Both <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZDQxY2ViYzQtNTRlMy00MGJiLTk1MDctZWUzMjI4NWFkM2Ex"--><a href="http://www.usn.pp.ru/downloads/java/usn-tomcat-redirect-valve/tomcat-redirect-valve-20110819-src.zip">the source</a> and a compiled <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZGY4ZmU2OWEtODg3ZS00ZTI2LWJmNDctZjExNDU4MjNkMWVh"--><a href="http://www.usn.pp.ru/downloads/java/usn-tomcat-redirect-valve/tomcat-redirect-valve-20110819.jar">.jar</a> are available. Still, as the code is not likely to become a part of the Tomcat project, the package name prefix is changed from <code>org.apache</code> to <code>usn.apache</code> .<br />
<br />
<h4>
Usage – HOW-TO</h4>
<br />
<ol style="margin-left: 0.7em; margin-top: 0em; padding-left: 0.7em; text-align: left;">
<li>Download the <!--a href="https://drive.google.com/uc?export=download&id=0B2nLiheurgV7ZGY4ZmU2OWEtODg3ZS00ZTI2LWJmNDctZjExNDU4MjNkMWVh"--><a href="http://www.usn.pp.ru/downloads/java/usn-tomcat-redirect-valve/tomcat-redirect-valve-20110819.jar">.jar file</a> <span style="font-size: 130%; line-height: 77%;">☺</span>.</li>
<li>Copy it to the <code>lib</code> folder of your Tomcat installation or to <a href="http://tomcat.apache.org/tomcat-7.0-doc/class-loader-howto.html">any other appropriate location</a> of your choice. You may also wish to have a look at <a href="http://www.mulesoft.com/tomcat-classpath">Tomcat classloader overview</a> at MuleSoft.</li>
<li>Register RedirectValve as a Valve with your Tomcat by adding an appropriate section to the contents of relevant <code><Host></code> element in your <code>conf/server.xml</code> file. Say, a task of redirecting/mapping a Tomcat web application to a virtual directory at your front-end server on the same machine might be configured like this:<br />
<pre style="background-color: #eeeeee;"><Valve
className="usn.apache.catalina.valves.RedirectValve"
from="^http://([^:]+):8080/<i>my-web-app</i>/(.*)$"
to="http://{1}/<i>my-virtual-dir-at-the-front-end-server</i>/{2}"
/>
</pre>
Feel free to add as many RedirectValve elements in a series as you wish.<br />
</li>
<li>Restart Tomcat.</li>
<li>Enjoy <span style="font-size: 130%; line-height: 77%;">☺</span>.</li>
</ol>
<br />
<hr />
<br />
<h4>
<a href="https://www.blogger.com/null" id="jboss-implementation" style="color: inherit; cursor: inherit; text-decoration: inherit;">"All we need is already here"</a></h4>
<span style="font-size: smaller;">added on 2011-08-24</span><br />
<br />
Being curious about when Google gets my blog indexed and listed <span style="font-size: 130%; line-height: 77%;">☺</span>, I have occasionally found somewhere in the third dozen of search result pages that a "Redirect Valve" for Tomcat already exists as <a href="http://docs.jboss.org/jbossweb/3.0.x/rewrite.html">RewriteValve</a>, a component of <a href="http://www.jboss.org/jbossweb">JBoss Web Server</a>. Looks like the code does not have any dependencies on JBoss internals, and the license is <a href="http://www.gnu.org/licenses/old-licenses/lgpl-2.1.html">LGPLv2.1</a> or later. The functionality looks really rich and closely resembles <a href="http://httpd.apache.org/docs/current/mod/mod_rewrite.html">mod_rewrite</a>.<br />
<br />
I wish I knew it before starting off with my simplistic implementation...<span style="font-size: 130%; line-height: 77%;">☺</span> Well, one more great tool to enjoy!<br />
<br />
<hr />
<br />
<h4>
<a href="https://www.blogger.com/null" id="tomcat8-implementation" style="color: inherit; cursor: inherit; text-decoration: inherit;">... now implemented as Tomcat 8 component</a></h4>
<span style="font-size: smaller;">added on 2013-03-17</span><br />
<br />
It recently appeared that a new rewrite valve implementation is being developed as a <a href="http://ci.apache.org/projects/tomcat/tomcat8/docs/rewrite.html">component of Tomcat 8</a>. The docs say that the capabilities and configuration facilities resemble Apache HTTP Server <a href="http://httpd.apache.org/docs/current/mod/mod_rewrite.html">mode_rewrite</a> closely. This is good news for all of us IMHO...<br />
<br />
<hr />
<br />
<h4>
<a href="https://www.blogger.com/null" id="tomcat8-release" style="color: inherit; cursor: inherit; text-decoration: inherit;">... now available as standard Tomcat feature</a></h4>
<span style="font-size: smaller;">added on 2014-11-16</span><br />
<br />
The latest and probably the final news on this topic is that as of June 2014 <a href="http://tomcat.apache.org/oldnews.html#Tomcat_8.0.9_Released">Tomcat 8 has reached release status</a>, with <a href="http://tomcat.apache.org/tomcat-8.0-doc/rewrite.html">rewrite valve</a> as a standard component. Nothing more left to be desired... <span style="font-size: 130%; line-height: 77%;">☺</span>Sergey Ushakovhttp://www.blogger.com/profile/13035264684517536678noreply@blogger.com2