Start line:  
End line:  

Snippet Preview

Snippet HTML Code

Stack Overflow Questions
We have a system where customers, mainly European enter texts (in UTF-8) that has to be distributed to different systems, most of them accepting UTF-8, but now we must also distribute the texts to a US system which only accepts US-Ascii 7-bit So now we'll need to translate all European characters to the nearest US-Ascii. Is there any Java libraries to help with this task? Right now we've just...
I must convert a char into a byte or a byte array. In other languages I know that a char is just a single byte. However, looking at the Java Character class, its min value is \u0000 and its max value is \uFFFF. This makes it seem like a char is 2 bytes long. Will I be able to store it as a byte or do I need to store it as two bytes? Before anyone asks, I will say that I'm trying to do th...
How do I truncate a java String so that I know it will fit in a given number of bytes storage once it is UTF-8 encoded?
The last week I've been trying to figure out why some stream decoding my newly adopted application is doing was giving me some major encoding problems. Finally I figured out that the problem was that the JARs/WAR being built with Ant and deployed to the server were being compiled with the javac task using the encoding UTF-8 instead of the system default of CP1252. This seems to be caused mainl...
in Taiwan we have a character encoding called "Unicode At One (UAO)", which is an extension to BIG-5 but is not supported by Java and Android. The code page is in http://moztw.org/docs/big5/table/uao241-b2u.txt My question is, how can I build a String object with byte array data, using this Charset? I guess I will extend the String class and do something in it, but I have no idea how to crea...
I've inherited a piece of code that makes intensive use of String -> byte[] conversions and vice versa for some homegrown serialisation code. Essentially the Java objects know how to convert their constituent parts into Strings which then get converted into a byte[]. Said byte array is then passed through JNI into C++ code that reconstitutes the byte[] into C++ std::strings and uses those to bo...
I'm using a legacy binary message format that requires a character sequence in ASCII-6 (6 bit ascii) encoding. I couldn't find a definition for ASCII-6 but they define the character mappings in their spec starting with A=0x01, B=0x02, etc. I'm wondering if there is an existing characterset in java for ASCII-6. If not can you create or define your own characterset somehow? If not is there a bet...
Let's say I have a String containing non ASCII characters in Java and I want to use String.format() to set it so the formatted String will have a minimum width regarding of the string's byte length. String s = "æøå"; String.format(l, "%" + 10 + "s" , s); This will result in a string with 7 leading white space. But what I want is there's should be only 4 leading white space since the origina...
First, I'd like to say that I've spent a lot of time searching for an explanation/solution. I've found hints of the problem, but no way to resolve my particular issue. Hence the post on a topic that seems to have been beaten to death in at least some cases. I have a Java test class that tests for proper encoding/decoding by a Mime utility. The strings used for testing are declared in the so...
I have an input ByteArrayOutputStream and need to convert that to a CharBuffer. I was trying to avoid creating a new string. Is there anyway to do this. I was trying to do the following, but I don't have the encoding for the string so the code below will not work (invalid output). ByteBuffer byteBuffer = ByteBuffer.wrap(byteOutputStream.toByteArray()); CharBuffer document = byteBuffer.asCha...
  /*
   * Copyright 2000-2006 Sun Microsystems, Inc.  All Rights Reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
   * under the terms of the GNU General Public License version 2 only, as
   * published by the Free Software Foundation.  Sun designates this
   * particular file as subject to the "Classpath" exception as provided
   * by Sun in the LICENSE file that accompanied this code.
  *
  * This code is distributed in the hope that it will be useful, but WITHOUT
  * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
  * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
  * version 2 for more details (a copy is included in the LICENSE file that
  * accompanied this code).
  *
  * You should have received a copy of the GNU General Public License version
  * 2 along with this work; if not, write to the Free Software Foundation,
  * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
  *
  * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
  * CA 95054 USA or visit www.sun.com if you need additional information or
  * have any questions.
  */
 
 // -- This file was mechanically generated: Do not edit! -- //
 
 package java.nio.charset;
 
 import java.nio.charset.CoderMalfunctionError;                  // javadoc
 

An engine that can transform a sequence of sixteen-bit Unicode characters into a sequence of bytes in a specific charset.

The input character sequence is provided in a character buffer or a series of such buffers. The output byte sequence is written to a byte buffer or a series of such buffers. An encoder should always be used by making the following sequence of method invocations, hereinafter referred to as an encoding operation:

  1. Reset the encoder via the reset method, unless it has not been used before;

  2. Invoke the encode method zero or more times, as long as additional input may be available, passing false for the endOfInput argument and filling the input buffer and flushing the output buffer between invocations;

  3. Invoke the encode method one final time, passing true for the endOfInput argument; and then

  4. Invoke the flush method so that the encoder can flush any internal state to the output buffer.

Each invocation of the encode method will encode as many characters as possible from the input buffer, writing the resulting bytes to the output buffer. The encode method returns when more input is required, when there is not enough room in the output buffer, or when an encoding error has occurred. In each case a CoderResult object is returned to describe the reason for termination. An invoker can examine this object and fill the input buffer, flush the output buffer, or attempt to recover from an encoding error, as appropriate, and try again.

There are two general types of encoding errors. If the input character sequence is not a legal sixteen-bit Unicode sequence then the input is considered malformed. If the input character sequence is legal but cannot be mapped to a valid byte sequence in the given charset then an unmappable character has been encountered.

How an encoding error is handled depends upon the action requested for that type of error, which is described by an instance of the CodingErrorAction class. The possible error actions are to CodingErrorAction.IGNORE the erroneous input, CodingErrorAction.REPORT the error to the invoker via the returned CoderResult object, or replace the erroneous input with the current value of the replacement byte array. The replacement is initially set to the encoder's default replacement, which often (but not always) has the initial value { (byte)'?' }; its value may be changed via the replaceWith method.

The default action for malformed-input and unmappable-character errors is to report them. The malformed-input error action may be changed via the onMalformedInput(java.nio.charset.CodingErrorAction) method; the unmappable-character action may be changed via the onUnmappableCharacter(java.nio.charset.CodingErrorAction) method.

This class is designed to handle many of the details of the encoding process, including the implementation of error actions. An encoder for a specific charset, which is a concrete subclass of this class, need only implement the abstract encodeLoop method, which encapsulates the basic encoding loop. A subclass that maintains internal state should, additionally, override the implFlush and implReset methods.

Instances of this class are not safe for use by multiple concurrent threads.

Author(s):
Mark Reinhold
JSR-51 Expert Group
Since:
1.4
See also:
java.nio.ByteBuffer
java.nio.CharBuffer
Charset
CharsetDecoder

public abstract class CharsetEncoder {
    private final Charset charset;
    private final float averageBytesPerChar;
    private final float maxBytesPerChar;
    private byte[] replacement;
        = .;
        = .;
    // Internal states
    //
    private static final int ST_RESET   = 0;
    private static final int ST_CODING  = 1;
    private static final int ST_END     = 2;
    private static final int ST_FLUSHED = 3;
    private int state = ;
    private static String stateNames[]
        = { "RESET""CODING""CODING_END""FLUSHED" };


    
Initializes a new encoder. The new encoder will have the given bytes-per-char and replacement values.

Parameters:
averageBytesPerChar A positive float value indicating the expected number of bytes that will be produced for each input character
maxBytesPerChar A positive float value indicating the maximum number of bytes that will be produced for each input character
replacement The initial replacement; must not be null, must have non-zero length, must not be longer than maxBytesPerChar, and must be legal
Throws:
java.lang.IllegalArgumentException If the preconditions on the parameters do not hold
    protected
                   float averageBytesPerChar,
                   float maxBytesPerChar,
                   byte[] replacement)
    {
        this. = cs;
        if (averageBytesPerChar <= 0.0f)
            throw new IllegalArgumentException("Non-positive "
                                               + "averageBytesPerChar");
        if (maxBytesPerChar <= 0.0f)
            throw new IllegalArgumentException("Non-positive "
                                               + "maxBytesPerChar");
        if (!Charset.atBugLevel("1.4")) {
            if (averageBytesPerChar > maxBytesPerChar)
                throw new IllegalArgumentException("averageBytesPerChar"
                                                   + " exceeds "
                                                   + "maxBytesPerChar");
        }
        this. = replacement;
        this. = averageBytesPerChar;
        this. = maxBytesPerChar;
        replaceWith(replacement);
    }

    
Initializes a new encoder. The new encoder will have the given bytes-per-char values and its replacement will be the byte array { (byte)'?' }.

Parameters:
averageBytesPerChar A positive float value indicating the expected number of bytes that will be produced for each input character
maxBytesPerChar A positive float value indicating the maximum number of bytes that will be produced for each input character
Throws:
java.lang.IllegalArgumentException If the preconditions on the parameters do not hold
    protected CharsetEncoder(Charset cs,
                             float averageBytesPerChar,
                             float maxBytesPerChar)
    {
        this(cs,
             averageBytesPerCharmaxBytesPerChar,
             new byte[] { (byte)'?' });
    }

    
Returns the charset that created this encoder.

Returns:
This encoder's charset
    public final Charset charset() {
        return ;
    }

    
Returns this encoder's replacement value.

Returns:
This encoder's current replacement, which is never null and is never empty
    public final byte[] replacement() {
        return ;
    }

    
Changes this encoder's replacement value.

This method invokes the implReplaceWith method, passing the new replacement, after checking that the new replacement is acceptable.

Parameters:
newReplacement The new replacement; must not be null, must have non-zero length, must not be longer than the value returned by the maxBytesPerChar method, and must be legal
Returns:
This encoder
Throws:
java.lang.IllegalArgumentException If the preconditions on the parameter do not hold
    public final CharsetEncoder replaceWith(byte[] newReplacement) {
        if (newReplacement == null)
            throw new IllegalArgumentException("Null replacement");
        int len = newReplacement.length;
        if (len == 0)
            throw new IllegalArgumentException("Empty replacement");
        if (len > )
            throw new IllegalArgumentException("Replacement too long");
        if (!isLegalReplacement(newReplacement))
            throw new IllegalArgumentException("Illegal replacement");
        this. = newReplacement;
        implReplaceWith(newReplacement);
        return this;
    }

    
Reports a change to this encoder's replacement value.

The default implementation of this method does nothing. This method should be overridden by encoders that require notification of changes to the replacement.

Parameters:
newReplacement
    protected void implReplaceWith(byte[] newReplacement) {
    }
    private WeakReference cachedDecoder = null;

    
Tells whether or not the given byte array is a legal replacement value for this encoder.

A replacement is legal if, and only if, it is a legal sequence of bytes in this encoder's charset; that is, it must be possible to decode the replacement into one or more sixteen-bit Unicode characters.

The default implementation of this method is not very efficient; it should generally be overridden to improve performance.

Parameters:
repl The byte array to be tested
Returns:
true if, and only if, the given byte array is a legal replacement value for this encoder
    public boolean isLegalReplacement(byte[] repl) {
        WeakReference wr = ;
        CharsetDecoder dec = null;
        if ((wr == null) || ((dec = (CharsetDecoder)wr.get()) == null)) {
            dec = charset().newDecoder();
            dec.onMalformedInput(.);
            dec.onUnmappableCharacter(.);
             = new WeakReference(dec);
        } else {
            dec.reset();
        }
        ByteBuffer bb = ByteBuffer.wrap(repl);
        CharBuffer cb = CharBuffer.allocate((int)(bb.remaining()
                                                  * dec.maxCharsPerByte()));
        CoderResult cr = dec.decode(bbcbtrue);
        return !cr.isError();
    }



    
Returns this encoder's current action for malformed-input errors.

Returns:
The current malformed-input action, which is never null
        return ;
    }

    
Changes this encoder's action for malformed-input errors.

This method invokes the implOnMalformedInput method, passing the new action.

Parameters:
newAction The new action; must not be null
Returns:
This encoder
Throws:
java.lang.IllegalArgumentException If the precondition on the parameter does not hold
    public final CharsetEncoder onMalformedInput(CodingErrorAction newAction) {
        if (newAction == null)
            throw new IllegalArgumentException("Null action");
         = newAction;
        implOnMalformedInput(newAction);
        return this;
    }

    
Reports a change to this encoder's malformed-input action.

The default implementation of this method does nothing. This method should be overridden by encoders that require notification of changes to the malformed-input action.

    protected void implOnMalformedInput(CodingErrorAction newAction) { }

    
Returns this encoder's current action for unmappable-character errors.

Returns:
The current unmappable-character action, which is never null
        return ;
    }

    
Changes this encoder's action for unmappable-character errors.

This method invokes the implOnUnmappableCharacter method, passing the new action.

Parameters:
newAction The new action; must not be null
Returns:
This encoder
Throws:
java.lang.IllegalArgumentException If the precondition on the parameter does not hold
                                                      newAction)
    {
        if (newAction == null)
            throw new IllegalArgumentException("Null action");
         = newAction;
        implOnUnmappableCharacter(newAction);
        return this;
    }

    
Reports a change to this encoder's unmappable-character action.

The default implementation of this method does nothing. This method should be overridden by encoders that require notification of changes to the unmappable-character action.

    protected void implOnUnmappableCharacter(CodingErrorAction newAction) { }

    
Returns the average number of bytes that will be produced for each character of input. This heuristic value may be used to estimate the size of the output buffer required for a given input sequence.

Returns:
The average number of bytes produced per character of input
    public final float averageBytesPerChar() {
        return ;
    }

    
Returns the maximum number of bytes that will be produced for each character of input. This value may be used to compute the worst-case size of the output buffer required for a given input sequence.

Returns:
The maximum number of bytes that will be produced per character of input
    public final float maxBytesPerChar() {
        return ;
    }

    
Encodes as many characters as possible from the given input buffer, writing the results to the given output buffer.

The buffers are read from, and written to, starting at their current positions. At most in.remaining() characters will be read and at most out.remaining() bytes will be written. The buffers' positions will be advanced to reflect the characters read and the bytes written, but their marks and limits will not be modified.

In addition to reading characters from the input buffer and writing bytes to the output buffer, this method returns a CoderResult object to describe its reason for termination:

  • CoderResult.UNDERFLOW indicates that as much of the input buffer as possible has been encoded. If there is no further input then the invoker can proceed to the next step of the encoding operation. Otherwise this method should be invoked again with further input.

  • CoderResult.OVERFLOW indicates that there is insufficient space in the output buffer to encode any more characters. This method should be invoked again with an output buffer that has more remaining bytes. This is typically done by draining any encoded bytes from the output buffer.

  • A malformed-input result indicates that a malformed-input error has been detected. The malformed characters begin at the input buffer's (possibly incremented) position; the number of malformed characters may be determined by invoking the result object's CoderResult.length() method. This case applies only if the malformed action of this encoder is CodingErrorAction.REPORT; otherwise the malformed input will be ignored or replaced, as requested.

  • An unmappable-character result indicates that an unmappable-character error has been detected. The characters that encode the unmappable character begin at the input buffer's (possibly incremented) position; the number of such characters may be determined by invoking the result object's length method. This case applies only if the unmappable action of this encoder is CodingErrorAction.REPORT; otherwise the unmappable character will be ignored or replaced, as requested.

In any case, if this method is to be reinvoked in the same encoding operation then care should be taken to preserve any characters remaining in the input buffer so that they are available to the next invocation.

The endOfInput parameter advises this method as to whether the invoker can provide further input beyond that contained in the given input buffer. If there is a possibility of providing additional input then the invoker should pass false for this parameter; if there is no possibility of providing further input then the invoker should pass true. It is not erroneous, and in fact it is quite common, to pass false in one invocation and later discover that no further input was actually available. It is critical, however, that the final invocation of this method in a sequence of invocations always pass true so that any remaining unencoded input will be treated as being malformed.

This method works by invoking the encodeLoop method, interpreting its results, handling error conditions, and reinvoking it as necessary.

Parameters:
in The input character buffer
out The output byte buffer
endOfInput true if, and only if, the invoker can provide no additional input characters beyond those in the given buffer
Returns:
A coder-result object describing the reason for termination
Throws:
java.lang.IllegalStateException If an encoding operation is already in progress and the previous step was an invocation neither of the reset method, nor of this method with a value of false for the endOfInput parameter, nor of this method with a value of true for the endOfInput parameter but a return value indicating an incomplete encoding operation
CoderMalfunctionError If an invocation of the encodeLoop method threw an unexpected exception
    public final CoderResult encode(CharBuffer inByteBuffer out,
                                    boolean endOfInput)
    {
        int newState = endOfInput ?  : ;
        if (( != ) && ( != )
            && !(endOfInput && ( == )))
            throwIllegalStateException(newState);
         = newState;
        for (;;) {
            CoderResult cr;
            try {
                cr = encodeLoop(inout);
            } catch (BufferUnderflowException x) {
                throw new CoderMalfunctionError(x);
            } catch (BufferOverflowException x) {
                throw new CoderMalfunctionError(x);
            }
            if (cr.isOverflow())
                return cr;
            if (cr.isUnderflow()) {
                if (endOfInput && in.hasRemaining()) {
                    cr = CoderResult.malformedForLength(in.remaining());
                    // Fall through to malformed-input case
                } else {
                    return cr;
                }
            }
            CodingErrorAction action = null;
            if (cr.isMalformed())
                action = ;
            else if (cr.isUnmappable())
                action = ;
            else
                assert false : cr.toString();
            if (action == .)
                return cr;
            if (action == .) {
                if (out.remaining() < .)
                    return .;
                out.put();
            }
            if ((action == .)
                || (action == .)) {
                // Skip erroneous input either way
                in.position(in.position() + cr.length());
                continue;
            }
            assert false;
        }
    }

    
Flushes this encoder.

Some encoders maintain internal state and may need to write some final bytes to the output buffer once the overall input sequence has been read.

Any additional output is written to the output buffer beginning at its current position. At most out.remaining() bytes will be written. The buffer's position will be advanced appropriately, but its mark and limit will not be modified.

If this method completes successfully then it returns CoderResult.UNDERFLOW. If there is insufficient room in the output buffer then it returns CoderResult.OVERFLOW. If this happens then this method must be invoked again, with an output buffer that has more room, in order to complete the current encoding operation.

If this encoder has already been flushed then invoking this method has no effect.

This method invokes the implFlush method to perform the actual flushing operation.

Parameters:
out The output byte buffer
Returns:
A coder-result object, either CoderResult.UNDERFLOW or CoderResult.OVERFLOW
Throws:
java.lang.IllegalStateException If the previous step of the current encoding operation was an invocation neither of the flush method nor of the three-argument encode(java.nio.CharBuffer,java.nio.ByteBuffer,boolean) method with a value of true for the endOfInput parameter
    public final CoderResult flush(ByteBuffer out) {
        if ( == ) {
            CoderResult cr = implFlush(out);
            if (cr.isUnderflow())
                 = ;
            return cr;
        }
        if ( != )
            throwIllegalStateException();
        return .// Already flushed
    }

    
Flushes this encoder.

The default implementation of this method does nothing, and always returns CoderResult.UNDERFLOW. This method should be overridden by encoders that may need to write final bytes to the output buffer once the entire input sequence has been read.

Parameters:
out The output byte buffer
Returns:
A coder-result object, either CoderResult.UNDERFLOW or CoderResult.OVERFLOW
    protected CoderResult implFlush(ByteBuffer out) {
        return .;
    }

    
Resets this encoder, clearing any internal state.

This method resets charset-independent state and also invokes the implReset method in order to perform any charset-specific reset actions.

Returns:
This encoder
    public final CharsetEncoder reset() {
        implReset();
         = ;
        return this;
    }

    
Resets this encoder, clearing any charset-specific internal state.

The default implementation of this method does nothing. This method should be overridden by encoders that maintain internal state.

    protected void implReset() { }

    
Encodes one or more characters into one or more bytes.

This method encapsulates the basic encoding loop, encoding as many characters as possible until it either runs out of input, runs out of room in the output buffer, or encounters an encoding error. This method is invoked by the encode method, which handles result interpretation and error recovery.

The buffers are read from, and written to, starting at their current positions. At most in.remaining() characters will be read, and at most out.remaining() bytes will be written. The buffers' positions will be advanced to reflect the characters read and the bytes written, but their marks and limits will not be modified.

This method returns a CoderResult object to describe its reason for termination, in the same manner as the encode method. Most implementations of this method will handle encoding errors by returning an appropriate result object for interpretation by the encode method. An optimized implementation may instead examine the relevant error action and implement that action itself.

An implementation of this method may perform arbitrary lookahead by returning CoderResult.UNDERFLOW until it receives sufficient input.

Parameters:
in The input character buffer
out The output byte buffer
Returns:
A coder-result object describing the reason for termination
    protected abstract CoderResult encodeLoop(CharBuffer in,
                                              ByteBuffer out);

    
Convenience method that encodes the remaining content of a single input character buffer into a newly-allocated byte buffer.

This method implements an entire encoding operation; that is, it resets this encoder, then it encodes the characters in the given character buffer, and finally it flushes this encoder. This method should therefore not be invoked if an encoding operation is already in progress.

Parameters:
in The input character buffer
Returns:
A newly-allocated byte buffer containing the result of the encoding operation. The buffer's position will be zero and its limit will follow the last byte written.
Throws:
java.lang.IllegalStateException If an encoding operation is already in progress
MalformedInputException If the character sequence starting at the input buffer's current position is not a legal sixteen-bit Unicode sequence and the current malformed-input action is CodingErrorAction.REPORT
UnmappableCharacterException If the character sequence starting at the input buffer's current position cannot be mapped to an equivalent byte sequence and the current unmappable-character action is CodingErrorAction.REPORT
    public final ByteBuffer encode(CharBuffer in)
        throws CharacterCodingException
    {
        int n = (int)(in.remaining() * averageBytesPerChar());
        ByteBuffer out = ByteBuffer.allocate(n);
        if ((n == 0) && (in.remaining() == 0))
            return out;
        reset();
        for (;;) {
            CoderResult cr = in.hasRemaining() ?
                encode(inouttrue) : .;
            if (cr.isUnderflow())
                cr = flush(out);
            if (cr.isUnderflow())
                break;
            if (cr.isOverflow()) {
                n = 2*n + 1;    // Ensure progress; n might be 0!
                ByteBuffer o = ByteBuffer.allocate(n);
                out.flip();
                o.put(out);
                out = o;
                continue;
            }
            cr.throwException();
        }
        out.flip();
        return out;
    }
    private boolean canEncode(CharBuffer cb) {
        if ( == )
            reset();
        else if ( != )
            throwIllegalStateException();
        CodingErrorAction ma = malformedInputAction();
        try {
            onMalformedInput(.);
            encode(cb);
        } catch (CharacterCodingException x) {
            return false;
        } finally {
            onMalformedInput(ma);
            onUnmappableCharacter(ua);
            reset();
        }
        return true;
    }

    
Tells whether or not this encoder can encode the given character.

This method returns false if the given character is a surrogate character; such characters can be interpreted only when they are members of a pair consisting of a high surrogate followed by a low surrogate. The canEncode(CharSequence) method may be used to test whether or not a character sequence can be encoded.

This method may modify this encoder's state; it should therefore not be invoked if an encoding operation is already in progress.

The default implementation of this method is not very efficient; it should generally be overridden to improve performance.

Returns:
true if, and only if, this encoder can encode the given character
Throws:
java.lang.IllegalStateException If an encoding operation is already in progress
    public boolean canEncode(char c) {
        CharBuffer cb = CharBuffer.allocate(1);
        cb.put(c);
        cb.flip();
        return canEncode(cb);
    }

    
Tells whether or not this encoder can encode the given character sequence.

If this method returns false for a particular character sequence then more information about why the sequence cannot be encoded may be obtained by performing a full encoding operation.

This method may modify this encoder's state; it should therefore not be invoked if an encoding operation is already in progress.

The default implementation of this method is not very efficient; it should generally be overridden to improve performance.

Returns:
true if, and only if, this encoder can encode the given character without throwing any exceptions and without performing any replacements
Throws:
java.lang.IllegalStateException If an encoding operation is already in progress
    public boolean canEncode(CharSequence cs) {
        CharBuffer cb;
        if (cs instanceof CharBuffer)
            cb = ((CharBuffer)cs).duplicate();
        else
            cb = CharBuffer.wrap(cs.toString());
        return canEncode(cb);
    }
    private void throwIllegalStateException(int fromint to) {
        throw new IllegalStateException("Current state = " + [from]
                                        + ", new state = " + [to]);
    }
New to GrepCode? Check out our FAQ X