To overcome above issue in kafka v 0.9.x consumer , set the "session.timeout.ms" property value @ consumer properties > group.min.session.timeout.ms property @ server.properties
Showing posts with label kafka. Show all posts
Showing posts with label kafka. Show all posts
Wednesday, July 6, 2016
Custom Serializer/Deserializer for Apache Kafka v 0.9.x
If we want to publish java beans to apache kafka v 0.9.x, we might need to write our own custom Serializer/Deserializer for them.
This custom serializers can be registered in consumer/producer properties.import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectInputStream;
import java.io.ObjectOutput;
import java.io.ObjectOutputStream;
import java.util.Map;
import org.apache.kafka.common.serialization.Deserializer;
import org.apache.kafka.common.serialization.Serializer;
import com.model.base.RawFile;
public class RawFileSerializer implements Serializer, Deserializer {
public RawFileSerializer() {
}
@Override
public RawFile deserialize(String arg0, byte[] fileContent) {
ByteArrayInputStream bis = new ByteArrayInputStream(fileContent);
ObjectInput in = null;
Object obj = null;
try {
in = new ObjectInputStream(bis);
obj = in.readObject();
} catch (IOException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
} finally {
try {
bis.close();
if (in != null) {
in.close();
}
} catch (IOException ex) {
// ignore close exception
}
}
return (RawFile) obj;
}
@Override
public void close() {
// TODO Auto-generated method stub
}
@Override
public void configure(Maparg0, boolean arg1) {
// TODO Auto-generated method stub
}
@Override
public byte[] serialize(String arg0, RawFile file) {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutput out = null;
byte[] rawFileBytes = null;
try {
out = new ObjectOutputStream(bos);
out.writeObject(file);
rawFileBytes = bos.toByteArray();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
if (out != null) {
out.close();
bos.close();
}
} catch (Exception ex) {
ex.getLocalizedMessage();
}
}
return rawFileBytes;
}
}
Wednesday, March 30, 2016
The session timeout is not within an acceptable range : Kafka v0.9.0.x
To overcome above issue [1] in the kafka 0.9.0.x, check following properties ;
- group.max.session.timeout.ms in the server.properties > session.timeout.ms in the consumer.properties
- group.min.session.timeout.ms in the server.properties < session.timeout.ms in the consumer.properties.
[1]
org.apache.kafka.common.errors.ApiException: The session timeout is not within an acceptable range.
Subscribe to:
Posts (Atom)