* Various fixes for Batch Source
1. Create intermediate topic/subscription prior to function running in case auto topic creation is turned off
2. Fix possible NPE in CronTrigger when calling stop()
3. Stop all producers created in ContextImpl
4. Delete intermediate topic for batch source
Co-authored-by: NJerry Peng <jerryp@splunk.com>

Motivation
close() method never gets called in BatchSource

Co-authored-by: NJerry Peng <jerryp@splunk.com>

Co-authored-by: NSanjeev Kulkarni <sanjeevk@splunk.com>

Fixes #5538

### Motivation



### Modifications
In function mode and sink mode, PulsarSource can read compacted topic。
By `inputSpecs` parameter, each topic can independently decide whether to read compacted

### Verifying this change
unit test:
org.apache.pulsar.io.PulsarFunctionE2ETest#testReadCompactedFunction
org.apache.pulsar.io.PulsarFunctionE2ETest#testReadCompactedSink

1）
Change the value of the CLI tool parameter "--custom-schema-input" from
"TopicName-> schemaType" 
 to
"topicName-> {" schemaType ":" type"," jsr310ConversionEnabled ": true," alwaysAllowNull ": true}"

2）
Modify Function.proto, add properties "jsr310ConversionEnabled", "alwaysAllowNull"。So that we can receive the above 2 parameters

3）
Modify the "FunctionConfigUtils#convert" method ， put the two parameters in "CustomSchemaInputs" into ConsumerSpec

4）
In “JavaInstanceRunnable#setupInput” method, put the above 2 parameters into "ConsumerConfig" and pass it to "PulsarSource", let it set the parameters into "schema" when creating the consumer of Source。
So that，The “function” can get these 2 parameters from the message of 
”currentRecord“

Fixes #7032

### Motivation
Source data flow:
Custom Source-> Function-> Pulsar producer-> Broker

Sink data flow:
Broker-> Pulsar consumer-> Function-> Custom Sink

Pulsar consumer is only used in sink mode, so it is necessary to add DLQ parameters for sink mode

* Moved ClassLoading and Reflections Helper functions to common

* Fix tests

* Fix test
Co-authored-by: NSanjeev Kulkarni <sanjeevk@splunk.com>

Co-authored-by: NSanjeev Kulkarni <sanjeevk@splunk.com>

Fixes #5503 
From #6445 

# Motivation
In functions, it will encounter ```ClassCastException``` when using the Avro schema for topics.

```
Exception in thread "main" java.lang.ClassCastException: org.apache.pulsar.shade.org.apache.avro.generic.GenericData$Record cannot be cast to io.streamnative.KeyValueSchemaTest$Foo2
	at io.streamnative.KeyValueSchemaTest.testConsumerByPythonProduce(KeyValueSchemaTest.java:412)
	at io.streamnative.KeyValueSchemaTest.main(KeyValueSchemaTest.java:305)
```

# Modifications
In functions, when using Avro schema specific the ClassLoader for ReflectDatumReader.

Add integration test ```testAvroSchemaFunction``` in class ```PulsarFunctionsTest```.

### Motivation

Master Issue: #5454 

When one Consumer subscribe multi topic, setSchemaInfoPorvider() will be covered by the consumer generated by the last topic.

### Modification
clone schema for each consumer generated by topic.
### Verifying this change
Add the schemaTest for it.

Fixes #5726 

### Motivation

When running pulsar-io connectors and functions from the Intellij IDE some actions fail
due to uncaught class-not-found throwables.
The expectation being that the class is being dynamically loaded and only the ClassNotFoundException will occur if the class is not found.
When the function is created or run with https://pulsar.apache.org/docs/en/functions-deploying/#local-run-mode this is indeed the case.
When running under the control https://pulsar.apache.org/docs/en/functions-debug/#debug-with-localrun-mode as a gradle plugin the class may already be known and throw a NoClassDefFoundError.
It seems to me that any time ClassNotFoundException is handled then NoClassDefFoundError should also be caught.

### Modifications

Wherever there was a `catch (ClassNotFoundException ` I replaced it with 
`catch (ClassNotFoundException | NoClassDefFoundError ` .
There were multiple cases where the ClassNotFoundException were handled e.g. the jar loader failed so the nar loader was used to handle the jar loader's failure.

Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>

* Add subscribe position param for consumer of sink
Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>

Fixes #5350

### Motivation

When using the `--output-serde-classname` option, `functionClassLoader` is not set correctly.

* Use classloading to load use code for functions

Upgrading to Mockito 2.28 and PowerMock 2.0. This a pre-step to be able to run CI with Java 11 / 12

* Functions use neg ack

* add neg ack for python functions

* adding tests

* refactoring maven project

* remove unnecessary edit

* fix pom due to maven bug

* Consolidating utils classes

* further refactoring

* adding missing class

* add license header

*Motivation*

Fixes #3925

We have 3 places of defining schema type enums. We kept adding
new schema type in pulsar-common. However we don't update the schema type
in wire protocol and schema storage.

This causes `SchemaType.NONE` is stored in SchemaRegistry.
It fails debeizum connector on restarting.

*Modifications*

Make sure all 3 places have consistent schema type definitions.
Record the correct schema type.

### Motivation
Fix #3741 

### Modifications
Support define not not allow null field in schema

### Verifying this change
Add not allow null field schema verify

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (yes)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

### Motivation
   when I study and read this module code , I fix some error like:
1. words error, such as JaveInstance -> JavaInstance, Recieved -> Received,  Unknwon -> Unknown, alterted -> alterted, Updare -> Update, etc.
2. remove not use import java class.
3. duplicated code in this  java class I extract method. 
4. javadoc  styles are consistent (not required)

### Modifications

minor fix up in pulsar-functions module

Fixes #3734

*Motivation*

Exception occurred when using `BytesSchema.of()`

```
Exception in thread "main" java.lang.ExceptionInInitializerError
	at org.apache.pulsar.examples.simple.ProducerExample.main(ProducerExample.java:32)
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at org.apache.pulsar.client.internal.ReflectionUtils.catchExceptions(ReflectionUtils.java:36)
	at org.apache.pulsar.client.internal.DefaultImplementation.newKeyValueSchema(DefaultImplementation.java:158)
	at org.apache.pulsar.client.api.Schema.<clinit>(Schema.java:123)
	... 1 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.pulsar.client.internal.DefaultImplementation.lambda$newKeyValueSchema$16(DefaultImplementation.java:160)
	at org.apache.pulsar.client.internal.ReflectionUtils.catchExceptions(ReflectionUtils.java:34)
	... 3 more
Caused by: java.lang.NullPointerException
	at org.apache.pulsar.client.impl.schema.KeyValueSchema.<init>(KeyValueSchema.java:68)
	... 9 more
```

The problem introduced because the weird class loading and reflection sequence.

When accessing `BytesSchema`, `BytesSchema` will try to initialize `Schema`. When initializing Schema, it will attempts
to initialize `KV_BYTES` using reflection, and initializing KV_BYTES requires `BytesSchema`. Hence it causes KV_BYTES not being
initialized correctly.

The change is to avoid this recursive class loading.

* Added ability to specify consumer queue size for function input topics

* fix and update PR

* fix function trigger

* simplifying code

* Cleanup consumer subscriptions and fix graceful shutdown for functions

* cleaning up

* removing testing files

* add unit tests

* adding integration testing

* refactoring

* refactoring and adding tests

* cleaning up

Clean up and correct properties to producer and consumers created by Functions/Sinks/Sources (#3315)

* clean up and correct properties to producer and consumers created by functions

* fix test

* cleaning up comment

* refactoring

change kafkaConnectSource to return KeyValue type record.

*Motivation*

Currently trigger function is broken due to the schema enforcement we added recently:
A producer without schema can't produce messages into a topic with schema.

*Changes*

- Rename `AUTO` to `AUTO_CONSUME`
- Introduce `AUTO_PRODUCE` schema. The schema produces `byte[]`, but it will validate
  the bytes are compatible with the schema associated with the topic before producing.
- Change trigger function to use `AUTO_PRODUCE` schema.

* Added ability to specify dead letter topic to functions

* Fix bug

* Added an example function that fails on a particular message consistently

* Revert change

*Motivation*

with more and more connector are added, it becomes expensive to start all external services at the begin.

*Changes*

- refactor the connector testing framework to start external service before methods
- fix kafka, cassandra and mysql connectors

* [schema] enable Schema.AUTO if functions or connectors are using GenericRecord

* add schema-api to functions-api

* Fix validator logic to differentiate between serializer and deserializer

* Expand to include backend and schema

* Fix buil

* Fix unittest

* Fixed unittest

Attach Producer/Consumer property tags so its easier to identify topics being produced/consumed by functions (#2490)

* Saved copy of work

* revert conf changes

* Prepare for the pr

* Addressed reviewer comments