Asked  4 Months ago    Answers:  5   Viewed   1.9k times

I am new to Kafka and trying to create a new topic on my local machine.

I am following this link.

Here are the steps which i followed:

  1. Start zookeeper

    bin/zookeeper-server-start.sh config/zookeeper.properties
    
  2. Start kafka-server

    bin/kafka-server-start.sh config/server.properties
    
  3. Create a topic

    bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
    

but when creating the topic, i am getting the following error:

Exception in thread "main" joptsimple.UnrecognizedOptionException: bootstrap-server is not a recognized option
    at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
    at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
    at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
    at joptsimple.OptionParser.parse(OptionParser.java:396)
    at kafka.admin.TopicCommand$TopicCommandOptions.<init>(TopicCommand.scala:358)
    at kafka.admin.TopicCommand$.main(TopicCommand.scala:44)
    at kafka.admin.TopicCommand.main(TopicCommand.scala)

Is there any other configuration required to create a topic? What wrong am i doing

 Answers

41

If you use version older than 2.2 you should use --zookeeper option and pass connection string to zookeeper

Command would be something like that:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Tuesday, August 10, 2021
 
Kenny
answered 4 Months ago
59

I found the error. Observing zookeeper logs when the server started I noticed:

server.1=mylocal-0.:2888:3888

with a dot (.) after the name of the host.

The script that produces the zookeeper's config is from https://github.com/kubernetes/contrib/blob/master/statefulsets/zookeeper/zkGenConfig.sh

Looking inside I see that DOMAIN is not filled:

HOST=`hostname -s`
DOMAIN=`hostname -d`

function print_servers() {
    for (( i=1; i<=$ZK_REPLICAS; i++ ))
    do
        echo "server.$i=$NAME-$((i-1)).$DOMAIN:$ZK_SERVER_PORT:$ZK_ELECTION_PORT"
    done
}

For my case (localhost) I don't need domain, so I removed that variable.

Now zookeeper and kafka communicate with no errors.

Tuesday, August 3, 2021
 
Gerardo
answered 4 Months ago
28

Here's the error:

Missing required configuration "confluent.topic.bootstrap.servers" which has no default value.

The problem is that you've taken the config for the HDFS Sink connector, and changed the connector for a different one (HDFS 3 Sink), and this one has different configuration requirements.

You can follow the quickstart for the HDFS 3 sink connector, or fix your existing configuration by adding

confluent.topic.bootstrap.servers=10.64.2.236:6667
confluent.topic.replication.factor=1

Note: in your example you missed the s from confluent.topic.bootstrap.servers which is why it didn't work

Wednesday, September 1, 2021
 
supermitch
answered 3 Months ago
92

The inline backend is set-up so that when each cell is finished executing, any matplotlib plot created in the cell will be displayed.

You are displaying your figure once using the display function, and then the figure is being displayed again automatically by the inline backend.

The easiest way to prevent this is to add plt.close() at the end of the code in your cell.

Wednesday, October 13, 2021
 
Quang Hoang
answered 2 Months ago
61

Figured it out. For some reason, some variables aren't picked up correctly from the environment. I noticed this yesterday with ZOOKEEPER_KERBEROS_REMOVEREALMFROMPRINCIPAL (and REMOVEHOSTFROMPRINCIPAL). So I tried moving these

"ZOOKEEPER_AUTHPROVIDER_1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider",
"ZOOKEEPER_REQUIRECLIENTAUTHSCHEME=SASL",

into

KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/zookeeper/secrets/zookeeper_jaas.conf -Dzookeeper.kerberos.removeHostFromPrincipal=true -Dzookeeper.kerberos.removeRealmFromPrincipal=true -Dzookeeper.authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider -Dzookeeper.requireClientAuthScheme=sasl"

And that sorted it.

Wednesday, October 27, 2021
 
BobtheMagicMoose
answered 1 Month ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :  
Share