African-Americans are a distinct ethnocultural group with their own cultural references, dialect, religious traditions, and so on (of course this doesn't apply to all black people; I'm talking about averages here.) "Black" in France is just a phenotype.
Progressive-leaning people in the US would typically be uncomfortable with the idea of "assimilating" Blacks because it implies that White Anglo-Saxon culture is (or should be) the default or standard, whereas in reality it has coexisted with Black culture in the same country for hundreds of years.
As a black man who went to almost entirely white schools my whole life and grew up in a very white place in New York I can sort of confirm what you're saying. When I went to college and met other black students who grew up around more black people I found that I had very little in common with them and it was hard to communicate. I was then, and always have been, more comfortable around white people.
On a side note, I was born in Jamaica and so when people refer to me as African American I always have a quiet chuckle to myself since I've never considered myself to be African and my passport says Jamaica.
I'm curious what the differences where in communication, here in the UK the black people I've worked with have been no harder to communicate with than anyone else, I have noticed there is slightly more of a communication gap with black people who come from London (but I've only known two and they where both deeply religious which I think had more to do with it).
Black Americans have had their own culture developing separately from Anglo-Saxon Americans for hundreds of years. I'm not sure why you think immigrants to Britain that don't share any common group characteristic other than "has dark skin and an African phenotype" are comparable.
The difference is that from 1066 until recently England has had one dominant ethnocultural group. Relatively small groups of immigrants are able to integrate into this culture seamlessly.
Black people in England aren't culturally "Black", they are either culturally English, or culturally Trinidadan, Jamaican, Ugandan, and so on. Or some combination/superposition of the above.
There is no particularly strong shared experience between a Trinidadan-English person and a Zimbabwean-English person other than living in England.
Blacks in the US, on the other hand, have a common and separate culture. 20th century immigrants are a tiny proportion of the (biologically) Black population of the US. The overwhelming majority of US black people are descended from slaves.
This population has been separated from the white population to varying degrees from the time of slavery to the present day. Now there is more mixing so the cultures are converging, but that can't happen overnight.
You may as well ask why Scotland and England have separate cultures, or England and China. It's not a matter of skin color; it's a matter of a group of people that have primarily had contact with each other, and less contact with other groups, for hundreds of years. I am not sure how else to explain it.
Edit: re-reading your comment. I didn't realize you were asking about communication difficulties specifically. This can probably be largely explained by the fact that African-Americans speak their own dialect of English that is in some ways quite dramatically different from the (White) standard. Here is some info: https://en.wikipedia.org/wiki/African_American_Vernacular_En...
I've definitely heard this before from African immigrants: they feel completely separate from "African-Americans", and frequently don't even like to be associated with them. Jamaica would be no different; you didn't grow up in the neighborhoods and with the same experiences that a black native-born American would have.
Progressive-leaning people in the US would typically be uncomfortable with the idea of "assimilating" Blacks because it implies that White Anglo-Saxon culture is (or should be) the default or standard, whereas in reality it has coexisted with Black culture in the same country for hundreds of years.